-- Logs begin at Mon 2022-07-04 09:28:21 UTC, end at Mon 2022-07-04 10:29:14 UTC. -- Jul 04 09:54:24.812442 dv-cinder-848418-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 04 09:54:24.823537 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 04 09:54:24.825573 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 4 09:54:24 2022] *** Jul 04 09:54:24.825573 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 04 09:54:24.825573 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 04 09:54:24.825573 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: nodename: dv-cinder-848418-1-iscsi Jul 04 09:54:24.825573 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: machine: x86_64 Jul 04 09:54:24.825573 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: clock source: unix Jul 04 09:54:24.825573 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: pcre jit disabled Jul 04 09:54:24.825573 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: detected number of CPU cores: 4 Jul 04 09:54:24.825573 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: current working directory: / Jul 04 09:54:24.825573 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: detected binary path: /usr/bin/uwsgi-core Jul 04 09:54:24.825573 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: your processes number limit is 47680 Jul 04 09:54:24.826157 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: your memory page size is 4096 bytes Jul 04 09:54:24.826157 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: detected max file descriptor number: 2048 Jul 04 09:54:24.826157 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: lock engine: pthread robust mutexes Jul 04 09:54:24.826157 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: thunder lock: enabled Jul 04 09:54:24.826157 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 04 09:54:24.826157 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 04 09:54:24.869343 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: Python main interpreter initialized at 0x56267f11f9c0 Jul 04 09:54:24.869343 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: python threads support enabled Jul 04 09:54:24.869343 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: your server socket listen backlog is limited to 100 connections Jul 04 09:54:24.869343 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: your mercy for graceful operations on workers is 90 seconds Jul 04 09:54:24.869768 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: mapped 403077 bytes (393 KB) for 2 cores Jul 04 09:54:24.869768 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: *** Operational MODE: preforking *** Jul 04 09:54:24.870085 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: *** uWSGI is running in multiple interpreter mode *** Jul 04 09:54:24.870085 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: spawned uWSGI master process (pid: 80086) Jul 04 09:54:24.870312 dv-cinder-848418-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 04 09:54:24.871845 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: spawned uWSGI worker 1 (pid: 80099, cores: 1) Jul 04 09:54:24.871845 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: spawned uWSGI worker 2 (pid: 80100, cores: 1) Jul 04 09:54:24.871845 dv-cinder-848418-1-iscsi devstack@keystone.service[80086]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 04 09:54:30.256017 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 09:54:30.256017 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: from cryptography.utils import int_from_bytes Jul 04 09:54:30.256840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 09:54:30.256840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: from cryptography.utils import int_from_bytes Jul 04 09:54:30.256949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 09:54:30.256949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: from cryptography.utils import int_from_bytes Jul 04 09:54:30.257363 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 09:54:30.257363 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: from cryptography.utils import int_from_bytes Jul 04 09:54:30.304815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 04 09:54:30.305013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 04 09:54:30.305229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] command line args: [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 04 09:54:30.305401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 04 09:54:30.305610 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ================================================================================ {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 04 09:54:30.305760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 04 09:54:30.305944 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] admin_token = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.306030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 04 09:54:30.306099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] command line args: [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 04 09:54:30.306164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] config_dir = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.306326 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.306418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 04 09:54:30.306485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] config_source = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.306549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ================================================================================ {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 04 09:54:30.306607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] debug = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.306760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] admin_token = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.306850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.307137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] config_dir = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.307137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.307347 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] config_source = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.307489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] debug = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.308561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.308561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.308561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.308561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.308561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.308879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.308941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.308998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.309054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_config_append = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.309111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.309381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_dir = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.309534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.309810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_file = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.310071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.310285 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.310486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.310657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.310891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_config_append = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.311053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.311248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.311248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_dir = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.311622 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_file = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.312064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.312064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.312409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.312603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.312795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.313039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.313244 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.313415 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.313572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.313572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.313892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.314603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.314788 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.314956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.315090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.315218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.315353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.315492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.315634 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.315769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.315905 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.316040 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.316040 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.316340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.79/identity {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.316481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.316610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] publish_errors = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.316736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.316870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.317004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.317139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.317139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.317378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.79/identity {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.317378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] publish_errors = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.317378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.317378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.317378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.318016 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.318172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.318316 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.318444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.318569 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] standard_threads = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.318763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] standard_threads = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.318928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.319097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.319226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.319359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.319492 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.319631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.319766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] use_journal = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.319900 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] use_journal = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.320038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] use_json = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.320165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] use_json = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.320291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] use_stderr = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.320425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] use_stderr = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.320553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] use_syslog = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.320680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] use_syslog = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.320807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.320933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:54:30.321058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.321193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.321335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.321470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.321597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.321765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.321910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.322046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.322180 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.322309 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.322434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.322566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.322693 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.322827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.322955 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.323080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.323205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.323331 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.323457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.323601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.323730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.323856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.323982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.324107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.324240 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.324379 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.324379 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.324635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.324762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.324894 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.325021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.325228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.325359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.325502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.325660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.325804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.325934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.326069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.326197 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.326323 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.326455 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.326583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.326710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.326848 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.326984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.327112 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.327254 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.327386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.327512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.327648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.327777 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.327913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.328041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.328168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.328302 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.328430 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.328557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.328684 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.328810 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.328947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.329087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.329215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.329341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.329470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.329974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.330419 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.330562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.330692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.330826 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.330954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.331081 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.331215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.331352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.331496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.331625 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.331758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.331893 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.332021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.332226 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.332357 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.332484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.332610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.332736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.332868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.333010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.333137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.333272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.333410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.333537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.333800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.333978 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.334117 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.334246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.334374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.334506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.334633 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.334767 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.334895 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.335021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.335156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.336599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.336769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.337003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.337167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.337356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.337552 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.337758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.338028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.338260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.338425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.338616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.338803 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.338997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.338997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.339404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.339571 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.339760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.339954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.340143 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.340378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.340586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.340788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.340952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.341189 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.341393 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.341552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.341766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.341953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.342095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.342224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.342389 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.342521 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.342655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.342790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.342927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.343086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.343216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.343216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.343456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.343592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.343731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.343867 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.344003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.344003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.344252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.344419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.344553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.344688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.344817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.344944 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.345070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.345275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.345791 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.345934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.346076 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.346205 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.346334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.346475 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.346613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.346742 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.346870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.347041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.347201 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.347343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.347472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.347599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.347726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.347870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.348003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.348139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.348267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.348404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.348533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.348660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.348787 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.348993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.349126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.349126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.349381 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.349510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.349655 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.349655 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.350265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.350413 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.350542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.350669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.350797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.350925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.351052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.351189 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.351327 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.351455 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.351583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.351716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.351850 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.352111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.352262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.352393 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.352527 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.352656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.352792 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.352922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.353057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.353186 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.353314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.353441 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.353577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.353774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.external = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.353906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.354044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.354172 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.external = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.354301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.354439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.354595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.354740 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.354877 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.password = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.355006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.355134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.token = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.355270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.password = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.355402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.355540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.token = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.355670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.355810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.355939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.356067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.356195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.356335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.356467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.356595 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.356724 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.356866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.356997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.357134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.357263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.357398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.357540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.357709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.357919 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.358076 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.358209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.358337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.358466 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.358594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.358722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.358858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.358987 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.359115 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.359242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.359370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.359508 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.359646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.359776 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.359905 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.360032 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.360161 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.360297 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.360433 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.360572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.360702 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.360867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.360867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.361586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.361755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.361898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.362027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.362155 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.362283 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.362412 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.362548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.362679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.362807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.362935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.363062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.363189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.363325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.363455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.363595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.363724 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.363852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.363980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.364116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.364246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.364374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.364503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.364631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.364761 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.364896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.365026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.365154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.365282 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.365419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.365660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.365801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.365930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.366058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.366186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.366313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.366453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.366583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.366711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.366845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.366974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.367101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.367254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.367385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.367514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.367641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.367769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.367897 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.368043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.368173 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.368303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.368430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.368558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.368685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.368821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.368952 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.369129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.369263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.369391 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.369552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.369715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.369847 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.369977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.370105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.370239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.370369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.370505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.370634 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.370761 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.370903 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.371033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.371160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.371296 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.371424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.371551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.371678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.371805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.371940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.372077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.372206 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.372334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.372461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.372588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.372725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.372862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.372991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.372991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.373233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.373360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.373487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.373640 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.373775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.373903 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.374030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.374165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.374306 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.374456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.374586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.374714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.374841 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.374970 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.375097 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.375235 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.375364 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.375502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.375630 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.375759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.375886 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.376022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.376151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.376291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.376455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.376738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.377227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.377372 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.377504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.377685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.377821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.377960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.378089 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.378227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.378368 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.378497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.378625 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.378765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.378894 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.379031 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.379160 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.379288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.379417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.379544 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.379680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.379809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.379937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.380065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.380192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.380320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.380466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.380597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.380725 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.380853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.380980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.381108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.381108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.381363 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.381491 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.381636 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.381771 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.381899 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.382038 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.382168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.382306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.382435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.382564 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.382691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.382827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.382957 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.383084 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.383212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.383339 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.383466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.383603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.383733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.383860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.383987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.384125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.384253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.384431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.384569 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.384700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.384828 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.384955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.384955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.385204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.385334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.385464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.385592 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.385782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.385922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.386063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.386063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.386306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.386435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.386562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.386562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.386819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.386949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.387077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.387205 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.387334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.387461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.387598 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.387738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.387868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.387996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.388128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.388260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.388398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.388529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.388663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.388792 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.388921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.389048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.389185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.389315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.389443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.389581 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.389731 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.389861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.390001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.390130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.390258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.390386 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.390514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.390641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.390777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.390907 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.391036 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.391164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.391291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.391428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] role.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.391635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.391766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] role.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.391766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.392031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.392237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.392683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.392683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.392993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.393125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.393254 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.393383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] role.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.393511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.393511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.393803 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] role.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.393803 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.394047 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.394047 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.394308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.394438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.394567 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.394695 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.394823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.394950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.395086 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.395217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.395360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.395490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.395618 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.395747 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.395885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.396015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.396143 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.396273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.396401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.396530 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.396667 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.396798 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.396926 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.397055 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.397193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.397324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.397462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.397593 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.397593 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.397987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.398151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.398291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.398436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.398566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.398703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.398838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.398972 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.399117 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.399261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.399398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.399533 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.399668 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.399803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.399945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.400082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.400217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.400351 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.400492 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.400627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.400770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.400917 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.401053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.401188 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.401323 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.401459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.401602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.401762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.401898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.402032 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.402167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.402302 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.402448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.402597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.402732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.402876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.403011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.403146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.403289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.403426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.403562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.403697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.403831 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.403966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.404109 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.404246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.404381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.404516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.404662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.404797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.404940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.405077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.405212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.405347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.405482 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.405672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.405826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.405963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.406098 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.406232 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.connection = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.406376 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.406522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.406670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.406805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.406948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.407084 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.407219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.407354 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.407489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.407693 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.408047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.408539 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.408539 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.408814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.408951 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.409095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.connection = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.409254 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.409405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.409542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.409701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.409841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.409976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.410111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.410258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.410395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.410531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.410666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.410801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.410938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.411081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.411229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.411365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.411500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.411635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.411770 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.411916 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.412053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 04 09:54:30.412191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.412191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.412191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.412191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.412191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.412191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.412191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.412191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:54:30.412191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 04 09:54:30.420988 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.421219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.421442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.422473 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.422711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.422946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.429964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.431309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.458627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.458876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.459340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.459574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.605180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.605534 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.653693 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.653961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.654295 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.654511 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.716833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.717149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.717303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.717303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.717593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.717762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.717900 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.718047 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.718186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.718336 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.718475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.718475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.743218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.743494 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.743666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.743811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.743960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.744099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.744236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.744384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.744522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.744682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.744819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.744955 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.745092 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.745239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.755578 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.755796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.768444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.768700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.768869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.769016 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.769168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.769307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.769490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.769691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.769873 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.770015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.770151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.770285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.770421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.770421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:54:30.786994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.787232 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 09:54:30.787384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.787525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 09:54:30.787679 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.787679 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 09:54:30.787941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.788078 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.788213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 09:54:30.788361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.788499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.788635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.788781 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.788916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.789051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.789051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.799415 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.799664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.799904 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.800141 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.800343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.800491 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.800631 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.800782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.808315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.808960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.811539 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.811825 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.812067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.812228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.812371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.812529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.812670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.812807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.812956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.813093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.813240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.813378 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.813553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.813721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.813883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.814028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.814263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.814499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.814710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.814866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.814866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.814866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.840014 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.841048 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.841585 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.841871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.842707 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.842991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.847272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.847526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.847945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.848282 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.848544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.848964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.855477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.855758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.856846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.857111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.860219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.860467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.860791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.861116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.861364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.861660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.861865 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.862015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.862155 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.862293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.862446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.862446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.862708 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.862996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.863270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.863485 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.880370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.880620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.882063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.882315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.885222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.886911 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.888791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.889088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.890550 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.890819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.895611 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.896119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.899290 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.899745 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.902923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.903222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.903376 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.903536 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.903682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.903816 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.903997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.904079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.904244 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.904324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.904535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.904815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.923702 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.923702 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.923887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.923968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.924067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.924148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.924367 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.924639 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.931937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.932451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.933599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.933890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.934189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.934463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.937157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.937440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.937746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.937872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.938032 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.938158 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.938262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.938344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.938577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.938867 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.951432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.951707 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.951986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.952126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.952213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.952298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.952528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.952655 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.952816 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.953060 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.954217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.954504 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.954818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.955207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.972279 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.972779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.975854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.976443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.979124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.979765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.980843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.981505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.982474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.982753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.983041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.983167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.983418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.983709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.990511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.991049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.992110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.992653 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:30.995300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.995585 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.995870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.995998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.996163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.996303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.996553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:30.996831 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.005229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:31.005514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.005817 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.006054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:31.006344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.006625 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.013544 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:31.013844 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.014121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.014395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.014517 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:54:31.014631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.014720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.014960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.015095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.015212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.015301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.015510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.015634 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.015842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.015966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.016086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.016191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.016358 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.016448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.016643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.016769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.016886 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.017045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.017290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.017412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.017754 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.018014 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.018288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:54:31.054949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56267f11f9c0 pid: 80099 (default app) Jul 04 09:54:31.055735 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56267f11f9c0 pid: 80100 (default app) Jul 04 09:54:31.058435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e050c-4cf8-44f1-8ce6-69f58937b5ab None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:31.058726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e050c-4cf8-44f1-8ce6-69f58937b5ab None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:31.059007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e050c-4cf8-44f1-8ce6-69f58937b5ab None None] PATH_INFO: `/v3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:31.063252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1/1] 10.222.0.79 () {50 vars in 791 bytes} [Mon Jul 4 09:54:31 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:38.770253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ba1fe-d712-462f-9f84-a9d91398e907 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:38.770851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ba1fe-d712-462f-9f84-a9d91398e907 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:38.770851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ba1fe-d712-462f-9f84-a9d91398e907 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:38.775162 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1/2] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:38.780762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6503120-639d-4b7d-8677-501e929548ee None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:38.781237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6503120-639d-4b7d-8677-501e929548ee None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:38.781327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6503120-639d-4b7d-8677-501e929548ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:38.850871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: INFO dbcounter [None req-e6503120-639d-4b7d-8677-501e929548ee None None] Registered counter for database keystone Jul 04 09:54:38.874466 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG oslo_db.sqlalchemy.engines [None req-e6503120-639d-4b7d-8677-501e929548ee None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80099) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 04 09:54:38.881457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writer thread running {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 04 09:54:39.055651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG passlib.handlers.bcrypt [None req-e6503120-639d-4b7d-8677-501e929548ee None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80099) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 04 09:54:39.056073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG passlib.handlers.bcrypt [None req-e6503120-639d-4b7d-8677-501e929548ee None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80099) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 04 09:54:39.110092 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e6503120-639d-4b7d-8677-501e929548ee None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:39.111729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e6503120-639d-4b7d-8677-501e929548ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:39.236280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 2/3] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 04 09:54:39.415325 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ba1fe-d712-462f-9f84-a9d91398e907 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:39.485867 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: INFO dbcounter [None req-ca8ba1fe-d712-462f-9f84-a9d91398e907 None None] Registered counter for database keystone Jul 04 09:54:39.505456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG oslo_db.sqlalchemy.engines [None req-ca8ba1fe-d712-462f-9f84-a9d91398e907 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80100) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 04 09:54:39.512234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writer thread running {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 04 09:54:39.629896 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ba1fe-d712-462f-9f84-a9d91398e907 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:39.634642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea095538-5e41-4df8-98a4-4fd4265744de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:39.635305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea095538-5e41-4df8-98a4-4fd4265744de admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:39.635591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea095538-5e41-4df8-98a4-4fd4265744de admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:39.635868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea095538-5e41-4df8-98a4-4fd4265744de admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:39.644283 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea095538-5e41-4df8-98a4-4fd4265744de admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:39.696659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ea095538-5e41-4df8-98a4-4fd4265744de admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 04 09:54:39.696888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea095538-5e41-4df8-98a4-4fd4265744de admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:39.707684 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 2/4] 10.222.0.79 () {58 vars in 1324 bytes} [Mon Jul 4 09:54:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:54:39.713898 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6503120-639d-4b7d-8677-501e929548ee None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:39.729260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6503120-639d-4b7d-8677-501e929548ee None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:39.734245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e550ebd-5569-4bb8-bd1e-16989ae3e550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:39.735379 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e550ebd-5569-4bb8-bd1e-16989ae3e550 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:39.735667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e550ebd-5569-4bb8-bd1e-16989ae3e550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:39.735985 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e550ebd-5569-4bb8-bd1e-16989ae3e550 admin admin] PATH_INFO: `/v3/projects/ca4e68f3b1bd4167b76136fe537c9f10` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:39.740198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e550ebd-5569-4bb8-bd1e-16989ae3e550 admin admin] RBAC: Authorizing `identity:get_project(project_id=ca4e68f3b1bd4167b76136fe537c9f10)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:39.790890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6e550ebd-5569-4bb8-bd1e-16989ae3e550 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 04 09:54:39.791488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e550ebd-5569-4bb8-bd1e-16989ae3e550 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:39.795949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 3/5] 10.222.0.79 () {56 vars in 1207 bytes} [Mon Jul 4 09:54:39 2022] GET /identity/v3/projects/ca4e68f3b1bd4167b76136fe537c9f10 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:41.784899 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c96da-c168-4fe6-bcd9-2cb4a6a76ee1 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:41.785765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c96da-c168-4fe6-bcd9-2cb4a6a76ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:41.785891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c96da-c168-4fe6-bcd9-2cb4a6a76ee1 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:41.792745 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 3/6] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:41.802147 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926ee0e-cb3b-43c7-a627-a34232993851 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:41.802620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926ee0e-cb3b-43c7-a627-a34232993851 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:41.802620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926ee0e-cb3b-43c7-a627-a34232993851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:41.823740 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d926ee0e-cb3b-43c7-a627-a34232993851 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:41.825359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d926ee0e-cb3b-43c7-a627-a34232993851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:41.835369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 4/7] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 04 09:54:41.861226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c96da-c168-4fe6-bcd9-2cb4a6a76ee1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:41.875315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c96da-c168-4fe6-bcd9-2cb4a6a76ee1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:41.880277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376ed0c-aa46-4677-9706-7b686edc217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:41.880949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376ed0c-aa46-4677-9706-7b686edc217e admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:41.881239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376ed0c-aa46-4677-9706-7b686edc217e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:41.881509 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376ed0c-aa46-4677-9706-7b686edc217e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:41.889754 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376ed0c-aa46-4677-9706-7b686edc217e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:41.891547 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376ed0c-aa46-4677-9706-7b686edc217e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:41.900608 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 4/8] 10.222.0.79 () {58 vars in 1324 bytes} [Mon Jul 4 09:54:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:54:41.906661 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926ee0e-cb3b-43c7-a627-a34232993851 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:41.916398 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926ee0e-cb3b-43c7-a627-a34232993851 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:41.922420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50489d-6920-4c31-b3d6-6e0d30bb95bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:41.923303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50489d-6920-4c31-b3d6-6e0d30bb95bf admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:41.923589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50489d-6920-4c31-b3d6-6e0d30bb95bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:41.924065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50489d-6920-4c31-b3d6-6e0d30bb95bf admin admin] PATH_INFO: `/v3/users/5875b90a72824e1280ee921aaccb7829` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:41.925725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50489d-6920-4c31-b3d6-6e0d30bb95bf admin admin] RBAC: Authorizing `identity:get_user(user_id=5875b90a72824e1280ee921aaccb7829)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:41.927597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50489d-6920-4c31-b3d6-6e0d30bb95bf admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:41.929180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 5/9] 10.222.0.79 () {56 vars in 1198 bytes} [Mon Jul 4 09:54:41 2022] GET /identity/v3/users/5875b90a72824e1280ee921aaccb7829 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:44.008765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526731e-3e8e-46b9-bad4-b93a3b6e1396 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:44.009359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526731e-3e8e-46b9-bad4-b93a3b6e1396 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:44.009359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526731e-3e8e-46b9-bad4-b93a3b6e1396 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:44.010715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 5/10] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:44.015925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bde339-f99e-43b3-b9e2-2e9490630743 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:44.016338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bde339-f99e-43b3-b9e2-2e9490630743 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:44.016521 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bde339-f99e-43b3-b9e2-2e9490630743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:44.034602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-51bde339-f99e-43b3-b9e2-2e9490630743 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:44.034796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ae7e1-7b8c-4745-97e6-a4732eb41702 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:44.034901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ae7e1-7b8c-4745-97e6-a4732eb41702 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:44.035144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ae7e1-7b8c-4745-97e6-a4732eb41702 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:44.035788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-51bde339-f99e-43b3-b9e2-2e9490630743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:44.036482 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 6/11] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:44.041420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1138b14-bd00-4fdc-b5cc-1235bc3064cc None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:44.041742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1138b14-bd00-4fdc-b5cc-1235bc3064cc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:44.042030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1138b14-bd00-4fdc-b5cc-1235bc3064cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:44.081316 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG passlib.handlers.bcrypt [None req-d1138b14-bd00-4fdc-b5cc-1235bc3064cc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80100) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 04 09:54:44.081773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG passlib.handlers.bcrypt [None req-d1138b14-bd00-4fdc-b5cc-1235bc3064cc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80100) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 04 09:54:44.120741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 6/12] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:54:44.128141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d1138b14-bd00-4fdc-b5cc-1235bc3064cc None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:44.129824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d1138b14-bd00-4fdc-b5cc-1235bc3064cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:44.143872 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bde339-f99e-43b3-b9e2-2e9490630743 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:44.169255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:44.232325 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 7/13] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:54:44.250562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bde339-f99e-43b3-b9e2-2e9490630743 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:44.254277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1138b14-bd00-4fdc-b5cc-1235bc3064cc None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:44.285023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb49dee-569d-4848-8059-d1e399675593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:44.285950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb49dee-569d-4848-8059-d1e399675593 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:44.286242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb49dee-569d-4848-8059-d1e399675593 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:44.286525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb49dee-569d-4848-8059-d1e399675593 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:44.298767 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb49dee-569d-4848-8059-d1e399675593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-5fb49dee-569d-4848-8059-d1e399675593 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:54:44.310028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:54:44.310927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:54:44.311755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:54:44.311755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:54:44.311755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:54:44.311755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:54:44.311755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:54:44.311755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:54:44.311755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:54:44.311755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:54:44.311755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:54:44.311755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:54:44.311755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:54:44.311755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:54:44.311755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 7/14] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:54:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:54:44.317565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb49dee-569d-4848-8059-d1e399675593 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:44.357421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1138b14-bd00-4fdc-b5cc-1235bc3064cc None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:44.359695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:44.396149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e396b-855b-4b24-a845-c5dd1d58b039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:44.397053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e396b-855b-4b24-a845-c5dd1d58b039 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:44.397338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e396b-855b-4b24-a845-c5dd1d58b039 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:44.397608 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e396b-855b-4b24-a845-c5dd1d58b039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:44.399619 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:44.419088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb49dee-569d-4848-8059-d1e399675593 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:44.451989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9037c-09bc-4c94-9746-b1a454e1f997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:44.452661 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9037c-09bc-4c94-9746-b1a454e1f997 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:44.452948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9037c-09bc-4c94-9746-b1a454e1f997 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:44.453228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9037c-09bc-4c94-9746-b1a454e1f997 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:44.454020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9037c-09bc-4c94-9746-b1a454e1f997 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:54:44.454330 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9037c-09bc-4c94-9746-b1a454e1f997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:44.455929 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9037c-09bc-4c94-9746-b1a454e1f997 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:44.465941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e396b-855b-4b24-a845-c5dd1d58b039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:44.467812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e396b-855b-4b24-a845-c5dd1d58b039 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:44.467921 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 8/15] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:54:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:54:44.473266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9037c-09bc-4c94-9746-b1a454e1f997 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:44.566942 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 8/16] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:54:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:54:44.571605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9037c-09bc-4c94-9746-b1a454e1f997 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:44.572908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e396b-855b-4b24-a845-c5dd1d58b039 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:44.576976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:44.607018 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2937187-f8ae-446f-b751-056125b817e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:44.607910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2937187-f8ae-446f-b751-056125b817e0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:44.608200 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2937187-f8ae-446f-b751-056125b817e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:44.609113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2937187-f8ae-446f-b751-056125b817e0 None admin] PATH_INFO: `/v3/users/5875b90a72824e1280ee921aaccb7829` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:44.609805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2937187-f8ae-446f-b751-056125b817e0 None admin] RBAC: Authorizing `identity:get_user(user_id=5875b90a72824e1280ee921aaccb7829)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:44.611533 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2937187-f8ae-446f-b751-056125b817e0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:44.613198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 9/17] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:54:44 2022] GET /identity/v3/users/5875b90a72824e1280ee921aaccb7829 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:54:44.618585 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2937187-f8ae-446f-b751-056125b817e0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:44.632716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:44.672709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e396b-855b-4b24-a845-c5dd1d58b039 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:44.708006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a27da-72d7-43d2-a993-2577349ff117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:44.708813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a27da-72d7-43d2-a993-2577349ff117 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:44.709100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a27da-72d7-43d2-a993-2577349ff117 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:44.709380 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a27da-72d7-43d2-a993-2577349ff117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:44.715434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2937187-f8ae-446f-b751-056125b817e0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:44.729718 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a27da-72d7-43d2-a993-2577349ff117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:44.731219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a27da-72d7-43d2-a993-2577349ff117 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:44.754456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fe472-875f-47e7-8b92-c6eee5b8eaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:44.755261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fe472-875f-47e7-8b92-c6eee5b8eaac None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-107a27da-72d7-43d2-a993-2577349ff117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:54:44.755489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:54:44.756335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:54:44.757111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:54:44.757111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:54:44.757111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:54:44.757111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:54:44.757321 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fe472-875f-47e7-8b92-c6eee5b8eaac None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:44.757321 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fe472-875f-47e7-8b92-c6eee5b8eaac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:44.757445 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 9/18] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:54:44.757511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fe472-875f-47e7-8b92-c6eee5b8eaac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:44.758441 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fe472-875f-47e7-8b92-c6eee5b8eaac None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:44.760044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 10/19] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:54:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:54:44.762619 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a27da-72d7-43d2-a993-2577349ff117 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:44.765447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fe472-875f-47e7-8b92-c6eee5b8eaac None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:44.822460 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:44.846469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:44.867296 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a27da-72d7-43d2-a993-2577349ff117 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:44.867437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fe472-875f-47e7-8b92-c6eee5b8eaac None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:44.900516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bacb24-a5c4-4be0-8c98-4456912aaebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:44.900718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11180d92-ba98-4fa8-92b4-b0012fae73df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:44.901181 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bacb24-a5c4-4be0-8c98-4456912aaebf None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:44.901377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11180d92-ba98-4fa8-92b4-b0012fae73df None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:44.901464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bacb24-a5c4-4be0-8c98-4456912aaebf None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:44.901696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11180d92-ba98-4fa8-92b4-b0012fae73df None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:44.901798 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bacb24-a5c4-4be0-8c98-4456912aaebf None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:44.901977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11180d92-ba98-4fa8-92b4-b0012fae73df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:44.902467 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bacb24-a5c4-4be0-8c98-4456912aaebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:54:44.902770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bacb24-a5c4-4be0-8c98-4456912aaebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:44.902861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11180d92-ba98-4fa8-92b4-b0012fae73df None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.domain.id=default, user.id=5875b90a72824e1280ee921aaccb7829) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:54:44.903109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11180d92-ba98-4fa8-92b4-b0012fae73df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:44.904165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bacb24-a5c4-4be0-8c98-4456912aaebf None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:44.904549 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11180d92-ba98-4fa8-92b4-b0012fae73df None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:44.911893 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 11/20] 10.222.0.79 () {56 vars in 1322 bytes} [Mon Jul 4 09:54:44 2022] GET /identity/v3/role_assignments?user.id=5875b90a72824e1280ee921aaccb7829&scope.domain.id=default&role.id=a3de9df253294d069dc1867fa26600c3 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:54:44.921960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 10/21] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:47.074707 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c9eee-0167-4382-9cc2-2910bf1143dd None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:47.075234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c9eee-0167-4382-9cc2-2910bf1143dd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:47.075306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c9eee-0167-4382-9cc2-2910bf1143dd None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:47.076796 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 12/22] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:47.091380 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0be9b-ae79-4792-85c4-de14d880943f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:47.091880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0be9b-ae79-4792-85c4-de14d880943f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:47.091960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0be9b-ae79-4792-85c4-de14d880943f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:47.127303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f5d0be9b-ae79-4792-85c4-de14d880943f None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:47.128650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f5d0be9b-ae79-4792-85c4-de14d880943f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:47.231074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 11/23] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:54:47.266290 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c9eee-0167-4382-9cc2-2910bf1143dd None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:47.410405 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c9eee-0167-4382-9cc2-2910bf1143dd None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:47.454463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048b524-be84-4658-86fe-f6ed02e3442d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:47.455915 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048b524-be84-4658-86fe-f6ed02e3442d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:47.456213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048b524-be84-4658-86fe-f6ed02e3442d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:47.456500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048b524-be84-4658-86fe-f6ed02e3442d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:47.470910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048b524-be84-4658-86fe-f6ed02e3442d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-e048b524-be84-4658-86fe-f6ed02e3442d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:54:47.479383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:54:47.480194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:54:47.480941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:54:47.480941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:54:47.480941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:54:47.480941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:54:47.480941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:54:47.480941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:54:47.480941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:54:47.480941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:54:47.480941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:54:47.480941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:54:47.480941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:54:47.480941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:54:47.480941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 13/24] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:54:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:54:47.487353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0be9b-ae79-4792-85c4-de14d880943f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:47.549476 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:47.609724 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0be9b-ae79-4792-85c4-de14d880943f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:47.648542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35752134-0e57-447b-b390-42ba292a00e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:47.649232 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35752134-0e57-447b-b390-42ba292a00e5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:47.649522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35752134-0e57-447b-b390-42ba292a00e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:47.649827 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35752134-0e57-447b-b390-42ba292a00e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:47.650599 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35752134-0e57-447b-b390-42ba292a00e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:54:47.650911 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35752134-0e57-447b-b390-42ba292a00e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:47.652337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35752134-0e57-447b-b390-42ba292a00e5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:47.670091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 12/25] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:54:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:54:47.676599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048b524-be84-4658-86fe-f6ed02e3442d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:47.691168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:47.751703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b885e-7714-43ab-9906-d38bfd41bec4 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:47.752327 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b885e-7714-43ab-9906-d38bfd41bec4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:47.752840 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b885e-7714-43ab-9906-d38bfd41bec4 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:47.755671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 13/26] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:47.759837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7daa2a-fb57-47a8-a750-da712e7ddfa4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:47.760344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7daa2a-fb57-47a8-a750-da712e7ddfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:47.760903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7daa2a-fb57-47a8-a750-da712e7ddfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:47.785236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7b7daa2a-fb57-47a8-a750-da712e7ddfa4 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:47.786986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7b7daa2a-fb57-47a8-a750-da712e7ddfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:47.864741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048b524-be84-4658-86fe-f6ed02e3442d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:47.882245 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 14/27] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:54:47.886086 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaa50b-516a-4cff-b31e-2023bae5993a None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:47.886602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaa50b-516a-4cff-b31e-2023bae5993a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:47.887075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaa50b-516a-4cff-b31e-2023bae5993a None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:47.889216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 15/28] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:47.894896 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa312c-2d38-472a-82ef-5e0b91f2ae80 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:47.895437 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa312c-2d38-472a-82ef-5e0b91f2ae80 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:47.895925 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa312c-2d38-472a-82ef-5e0b91f2ae80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:47.925863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b4aa312c-2d38-472a-82ef-5e0b91f2ae80 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:47.929187 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b4aa312c-2d38-472a-82ef-5e0b91f2ae80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:47.940462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dba353-96b6-454d-8bb6-dcb14d9df03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:47.941760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dba353-96b6-454d-8bb6-dcb14d9df03d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:47.942306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dba353-96b6-454d-8bb6-dcb14d9df03d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:47.942826 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dba353-96b6-454d-8bb6-dcb14d9df03d None admin] PATH_INFO: `/v3/users/5875b90a72824e1280ee921aaccb7829` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:47.944312 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dba353-96b6-454d-8bb6-dcb14d9df03d None admin] RBAC: Authorizing `identity:get_user(user_id=5875b90a72824e1280ee921aaccb7829)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:47.946223 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dba353-96b6-454d-8bb6-dcb14d9df03d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:47.948725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 14/29] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:54:47 2022] GET /identity/v3/users/5875b90a72824e1280ee921aaccb7829 => generated 247 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:47.949932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dba353-96b6-454d-8bb6-dcb14d9df03d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:47.977723 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:48.026515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 16/30] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:54:48.027961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa312c-2d38-472a-82ef-5e0b91f2ae80 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:48.029777 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:48.060126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dba353-96b6-454d-8bb6-dcb14d9df03d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:48.094219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01b723-e9b1-46e5-a44d-4d1cb577ac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:48.095016 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01b723-e9b1-46e5-a44d-4d1cb577ac10 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:48.095632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01b723-e9b1-46e5-a44d-4d1cb577ac10 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:48.095736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01b723-e9b1-46e5-a44d-4d1cb577ac10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:48.106448 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01b723-e9b1-46e5-a44d-4d1cb577ac10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:48.107949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01b723-e9b1-46e5-a44d-4d1cb577ac10 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-7c01b723-e9b1-46e5-a44d-4d1cb577ac10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:54:48.126751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:54:48.127680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:54:48.128937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:54:48.128937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:54:48.128937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:54:48.128937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:54:48.128937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 15/31] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:54:48.133901 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9d521-0070-45cd-9529-1294bda45485 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:48.134196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9d521-0070-45cd-9529-1294bda45485 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:48.134484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9d521-0070-45cd-9529-1294bda45485 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:48.135838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 16/32] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:48.136691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9d521-0070-45cd-9529-1294bda45485 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:48.147160 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa312c-2d38-472a-82ef-5e0b91f2ae80 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:48.185522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98178594-84f6-4b43-a907-94744b4a7011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:48.186673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98178594-84f6-4b43-a907-94744b4a7011 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:48.187151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98178594-84f6-4b43-a907-94744b4a7011 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:48.187619 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98178594-84f6-4b43-a907-94744b4a7011 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:48.189597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98178594-84f6-4b43-a907-94744b4a7011 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:48.191424 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98178594-84f6-4b43-a907-94744b4a7011 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:48.193381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 17/33] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:54:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:48.194642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98178594-84f6-4b43-a907-94744b4a7011 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:48.233566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:48.255300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9d521-0070-45cd-9529-1294bda45485 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:48.279590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:48.291366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eba36-79f9-48e9-b93c-ab26a7edaf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:48.292157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eba36-79f9-48e9-b93c-ab26a7edaf00 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:48.292447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eba36-79f9-48e9-b93c-ab26a7edaf00 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:48.292720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eba36-79f9-48e9-b93c-ab26a7edaf00 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:48.293470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8eba36-79f9-48e9-b93c-ab26a7edaf00 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:48.294914 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8eba36-79f9-48e9-b93c-ab26a7edaf00 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:48.311128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98178594-84f6-4b43-a907-94744b4a7011 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:48.349122 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade60e0e-2185-48a7-995d-e526c1350bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:48.350221 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade60e0e-2185-48a7-995d-e526c1350bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:48.351102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade60e0e-2185-48a7-995d-e526c1350bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:48.351627 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade60e0e-2185-48a7-995d-e526c1350bf9 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:48.352551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade60e0e-2185-48a7-995d-e526c1350bf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:54:48.353095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade60e0e-2185-48a7-995d-e526c1350bf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:48.354798 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade60e0e-2185-48a7-995d-e526c1350bf9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:48.368256 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 18/34] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:48.369945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933443d-91f3-4bc9-b6e6-d7d9cc94a9df None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:48.370442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933443d-91f3-4bc9-b6e6-d7d9cc94a9df None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:48.370954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933443d-91f3-4bc9-b6e6-d7d9cc94a9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:48.389298 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5933443d-91f3-4bc9-b6e6-d7d9cc94a9df None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:48.390822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5933443d-91f3-4bc9-b6e6-d7d9cc94a9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:48.448719 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 17/35] 10.222.0.79 () {60 vars in 1131 bytes} [Mon Jul 4 09:54:48 2022] POST /identity/v3/roles => generated 225 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:54:48.450145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eba36-79f9-48e9-b93c-ab26a7edaf00 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:48.472327 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 19/36] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:54:48.476957 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933443d-91f3-4bc9-b6e6-d7d9cc94a9df None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:48.534836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:48.652976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eba36-79f9-48e9-b93c-ab26a7edaf00 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:48.695125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933443d-91f3-4bc9-b6e6-d7d9cc94a9df None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:48.697262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72e0ea-5d3f-4d3f-821a-a1d6d06b2592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:48.698319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72e0ea-5d3f-4d3f-821a-a1d6d06b2592 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:48.698608 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72e0ea-5d3f-4d3f-821a-a1d6d06b2592 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:48.698952 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72e0ea-5d3f-4d3f-821a-a1d6d06b2592 None admin] PATH_INFO: `/v3/domains/default/users/5875b90a72824e1280ee921aaccb7829/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:48.701112 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72e0ea-5d3f-4d3f-821a-a1d6d06b2592 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5875b90a72824e1280ee921aaccb7829, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:48.702748 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72e0ea-5d3f-4d3f-821a-a1d6d06b2592 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:48.724707 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 18/37] 10.222.0.79 () {58 vars in 1361 bytes} [Mon Jul 4 09:54:48 2022] PUT /identity/v3/domains/default/users/5875b90a72824e1280ee921aaccb7829/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:54:48.726151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72e0ea-5d3f-4d3f-821a-a1d6d06b2592 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:48.737484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ed04-dceb-479b-ad81-2737b4919df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:48.738583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ed04-dceb-479b-ad81-2737b4919df2 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:48.739059 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ed04-dceb-479b-ad81-2737b4919df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:48.739522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ed04-dceb-479b-ad81-2737b4919df2 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:48.740483 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ed04-dceb-479b-ad81-2737b4919df2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:48.742271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ed04-dceb-479b-ad81-2737b4919df2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:48.787036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:48.868074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72e0ea-5d3f-4d3f-821a-a1d6d06b2592 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:48.899630 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 20/38] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:54:48 2022] POST /identity/v3/projects => generated 301 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:54:48.901255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd79df-27f7-44d1-b4b8-d67aacb7bea8 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:48.901557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd79df-27f7-44d1-b4b8-d67aacb7bea8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:48.901881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd79df-27f7-44d1-b4b8-d67aacb7bea8 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:48.903587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 21/39] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:48.904921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6fb2f-cb0b-4ec9-a4f2-4ab8255243dd None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:48.905215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6fb2f-cb0b-4ec9-a4f2-4ab8255243dd None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:48.905502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6fb2f-cb0b-4ec9-a4f2-4ab8255243dd None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:48.909213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 22/40] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:48.911762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6e8eb-923c-4a00-8f04-1c3080ba2393 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:48.912348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6e8eb-923c-4a00-8f04-1c3080ba2393 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:48.913560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6e8eb-923c-4a00-8f04-1c3080ba2393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:48.919932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab42f80-eb2e-46dd-86c4-bc9c15947757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:48.920655 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab42f80-eb2e-46dd-86c4-bc9c15947757 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:48.920949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab42f80-eb2e-46dd-86c4-bc9c15947757 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:48.921231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab42f80-eb2e-46dd-86c4-bc9c15947757 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:48.922091 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab42f80-eb2e-46dd-86c4-bc9c15947757 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:48.923760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab42f80-eb2e-46dd-86c4-bc9c15947757 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:48.945378 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-79a6e8eb-923c-4a00-8f04-1c3080ba2393 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:48.947979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-79a6e8eb-923c-4a00-8f04-1c3080ba2393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:48.951730 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 19/41] 10.222.0.79 () {60 vars in 1131 bytes} [Mon Jul 4 09:54:48 2022] POST /identity/v3/roles => generated 219 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:54:48.957681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e67add-743a-4cbd-b469-4e4d38c7f79f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:48.957681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e67add-743a-4cbd-b469-4e4d38c7f79f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:48.957681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e67add-743a-4cbd-b469-4e4d38c7f79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:48.978933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d2e67add-743a-4cbd-b469-4e4d38c7f79f None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:48.981142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d2e67add-743a-4cbd-b469-4e4d38c7f79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:49.054860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 23/42] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:54:49.078570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6e8eb-923c-4a00-8f04-1c3080ba2393 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:49.083667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 20/43] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:54:49.091397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:49.109289 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e67add-743a-4cbd-b469-4e4d38c7f79f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:49.146471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:49.196697 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6e8eb-923c-4a00-8f04-1c3080ba2393 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:49.241800 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e67add-743a-4cbd-b469-4e4d38c7f79f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:49.248838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6848e7-76a3-4246-aed6-0725a3a97fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:49.249551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6848e7-76a3-4246-aed6-0725a3a97fa9 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:49.249863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6848e7-76a3-4246-aed6-0725a3a97fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:49.250151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6848e7-76a3-4246-aed6-0725a3a97fa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:49.251022 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6848e7-76a3-4246-aed6-0725a3a97fa9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:49.252773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6848e7-76a3-4246-aed6-0725a3a97fa9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:49.273758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 24/44] 10.222.0.79 () {60 vars in 1131 bytes} [Mon Jul 4 09:54:49 2022] POST /identity/v3/roles => generated 223 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:54:49.398748 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eababbe5-6f77-4248-8c94-b84703fb7467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:49.399582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eababbe5-6f77-4248-8c94-b84703fb7467 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:49.399865 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eababbe5-6f77-4248-8c94-b84703fb7467 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:49.400138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eababbe5-6f77-4248-8c94-b84703fb7467 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:49.401425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eababbe5-6f77-4248-8c94-b84703fb7467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:49.402982 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eababbe5-6f77-4248-8c94-b84703fb7467 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:49.404878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 21/45] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:54:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:54:49.410783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6848e7-76a3-4246-aed6-0725a3a97fa9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:49.488333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:49.526014 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6848e7-76a3-4246-aed6-0725a3a97fa9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:49.559489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4687fd-fa26-42cc-8961-e411aca88e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:49.560155 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4687fd-fa26-42cc-8961-e411aca88e00 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:49.560444 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4687fd-fa26-42cc-8961-e411aca88e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:49.560752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4687fd-fa26-42cc-8961-e411aca88e00 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:49.561469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4687fd-fa26-42cc-8961-e411aca88e00 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:49.562988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4687fd-fa26-42cc-8961-e411aca88e00 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:49.592620 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 25/46] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:54:49 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:54:51.024457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b699ddd-0d52-4d1a-a3c6-52db71051071 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:51.025005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b699ddd-0d52-4d1a-a3c6-52db71051071 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:51.025005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b699ddd-0d52-4d1a-a3c6-52db71051071 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:51.026443 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 22/47] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:51.031964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97b22a-d92b-4701-80a0-f69f51a5dced None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:51.032379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97b22a-d92b-4701-80a0-f69f51a5dced None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:51.032577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97b22a-d92b-4701-80a0-f69f51a5dced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:51.051223 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ab97b22a-d92b-4701-80a0-f69f51a5dced None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:51.052416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ab97b22a-d92b-4701-80a0-f69f51a5dced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:51.126652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 26/48] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:54:51.151565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b699ddd-0d52-4d1a-a3c6-52db71051071 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:51.260885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b699ddd-0d52-4d1a-a3c6-52db71051071 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:51.262494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:51.297843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c6048-31fb-4ecb-86c0-2f273563321b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:51.298827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c6048-31fb-4ecb-86c0-2f273563321b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:51.299117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c6048-31fb-4ecb-86c0-2f273563321b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:51.299383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c6048-31fb-4ecb-86c0-2f273563321b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:51.310128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c6048-31fb-4ecb-86c0-2f273563321b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-0a0c6048-31fb-4ecb-86c0-2f273563321b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:54:51.318324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:54:51.319164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:54:51.320003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:54:51.320003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:54:51.320003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:54:51.320003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:54:51.320003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:54:51.320003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:54:51.320003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:54:51.320003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:54:51.320003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:54:51.320003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:54:51.320003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:54:51.320003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:54:51.320003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 23/49] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:54:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:54:51.326722 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97b22a-d92b-4701-80a0-f69f51a5dced None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:51.381661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:51.434120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97b22a-d92b-4701-80a0-f69f51a5dced None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:51.579705 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8f6c8-a279-4a0e-830a-2ea6c7bc4b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:51.580297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8f6c8-a279-4a0e-830a-2ea6c7bc4b16 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:51.580589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8f6c8-a279-4a0e-830a-2ea6c7bc4b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:51.580872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8f6c8-a279-4a0e-830a-2ea6c7bc4b16 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:51.581661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8f6c8-a279-4a0e-830a-2ea6c7bc4b16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:54:51.581975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8f6c8-a279-4a0e-830a-2ea6c7bc4b16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:51.583375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8f6c8-a279-4a0e-830a-2ea6c7bc4b16 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:51.591910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 27/50] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:54:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:51.598009 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c6048-31fb-4ecb-86c0-2f273563321b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:51.642586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edc4e4-f262-4a37-bc21-08d444b52fda None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:51.642965 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edc4e4-f262-4a37-bc21-08d444b52fda None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:51.643258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edc4e4-f262-4a37-bc21-08d444b52fda None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:51.644614 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 28/51] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:54:51.649696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acd40a-774d-4513-ae9a-4fdb0ce3772f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:51.649987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acd40a-774d-4513-ae9a-4fdb0ce3772f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:51.650265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acd40a-774d-4513-ae9a-4fdb0ce3772f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:51.666899 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f5acd40a-774d-4513-ae9a-4fdb0ce3772f None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:51.668098 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f5acd40a-774d-4513-ae9a-4fdb0ce3772f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:51.704221 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c6048-31fb-4ecb-86c0-2f273563321b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:51.733159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 29/52] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:54:51.737605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf9cc7-9ac8-43b2-8714-4d667395975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:51.738515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf9cc7-9ac8-43b2-8714-4d667395975c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:51.738801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf9cc7-9ac8-43b2-8714-4d667395975c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:51.739083 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf9cc7-9ac8-43b2-8714-4d667395975c None admin] PATH_INFO: `/v3/users/5875b90a72824e1280ee921aaccb7829` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:51.740285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf9cc7-9ac8-43b2-8714-4d667395975c None admin] RBAC: Authorizing `identity:get_user(user_id=5875b90a72824e1280ee921aaccb7829)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:51.741785 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf9cc7-9ac8-43b2-8714-4d667395975c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:51.743413 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 24/53] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:54:51 2022] GET /identity/v3/users/5875b90a72824e1280ee921aaccb7829 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:51.748774 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acd40a-774d-4513-ae9a-4fdb0ce3772f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:51.755920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf9cc7-9ac8-43b2-8714-4d667395975c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:51.787164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:51.793659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:51.853909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acd40a-774d-4513-ae9a-4fdb0ce3772f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:51.858664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf9cc7-9ac8-43b2-8714-4d667395975c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:51.887011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a841f-011d-41c8-b1b8-cec5abbbf9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:51.887823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a841f-011d-41c8-b1b8-cec5abbbf9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:51.888112 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a841f-011d-41c8-b1b8-cec5abbbf9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:51.888395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a841f-011d-41c8-b1b8-cec5abbbf9d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:51.889588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a841f-011d-41c8-b1b8-cec5abbbf9d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:51.891127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a841f-011d-41c8-b1b8-cec5abbbf9d8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:51.892427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4c78b-0363-4723-a7c3-19e050e9003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:51.893079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 30/54] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:54:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:51.893230 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4c78b-0363-4723-a7c3-19e050e9003e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:51.893517 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4c78b-0363-4723-a7c3-19e050e9003e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:51.893830 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4c78b-0363-4723-a7c3-19e050e9003e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:51.895027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4c78b-0363-4723-a7c3-19e050e9003e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:51.896502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4c78b-0363-4723-a7c3-19e050e9003e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:51.898193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 25/55] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:54:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:54:51.898684 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a841f-011d-41c8-b1b8-cec5abbbf9d8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:51.903386 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4c78b-0363-4723-a7c3-19e050e9003e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:51.997079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:51.997265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a841f-011d-41c8-b1b8-cec5abbbf9d8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:52.001028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4c78b-0363-4723-a7c3-19e050e9003e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:52.008188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:52.033075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964dd1aa-a003-4053-8e18-bfe719f04a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:52.033838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dd1aa-a003-4053-8e18-bfe719f04a13 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:52.034053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dd1aa-a003-4053-8e18-bfe719f04a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:52.034332 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dd1aa-a003-4053-8e18-bfe719f04a13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:52.035178 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964dd1aa-a003-4053-8e18-bfe719f04a13 None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.domain.id=default, user.id=5875b90a72824e1280ee921aaccb7829) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:54:52.035489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964dd1aa-a003-4053-8e18-bfe719f04a13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:52.036264 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68417dbe-a713-4c83-9a8b-275b1d846ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:52.036937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964dd1aa-a003-4053-8e18-bfe719f04a13 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:52.037027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68417dbe-a713-4c83-9a8b-275b1d846ff1 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:52.037260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68417dbe-a713-4c83-9a8b-275b1d846ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:52.037536 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68417dbe-a713-4c83-9a8b-275b1d846ff1 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:52.038293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68417dbe-a713-4c83-9a8b-275b1d846ff1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:52.039764 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68417dbe-a713-4c83-9a8b-275b1d846ff1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:52.045729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 31/56] 10.222.0.79 () {56 vars in 1322 bytes} [Mon Jul 4 09:54:51 2022] GET /identity/v3/role_assignments?user.id=5875b90a72824e1280ee921aaccb7829&scope.domain.id=default&role.id=a3de9df253294d069dc1867fa26600c3 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:54:52.072293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 26/57] 10.222.0.79 () {60 vars in 1140 bytes} [Mon Jul 4 09:54:51 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:54:54.132341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7cf36-d2d0-4cf6-83eb-4d2a58cab279 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:54.132901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7cf36-d2d0-4cf6-83eb-4d2a58cab279 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:54.133012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7cf36-d2d0-4cf6-83eb-4d2a58cab279 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:54.134737 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 32/58] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:54.142748 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de73aa-475f-4c4b-ade5-32805a082d1d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:54.143166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de73aa-475f-4c4b-ade5-32805a082d1d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:54.143271 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de73aa-475f-4c4b-ade5-32805a082d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:54.161647 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-62de73aa-475f-4c4b-ade5-32805a082d1d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:54.162845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-62de73aa-475f-4c4b-ade5-32805a082d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:54.237603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 27/59] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:54:54.263169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7cf36-d2d0-4cf6-83eb-4d2a58cab279 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:54.385248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7cf36-d2d0-4cf6-83eb-4d2a58cab279 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:54.420008 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821f5dc-e190-4c57-8947-b744acd18801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:54.420805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821f5dc-e190-4c57-8947-b744acd18801 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:54.421093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821f5dc-e190-4c57-8947-b744acd18801 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:54.421375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821f5dc-e190-4c57-8947-b744acd18801 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:54.422636 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821f5dc-e190-4c57-8947-b744acd18801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:54.424239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821f5dc-e190-4c57-8947-b744acd18801 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:54.425901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 33/60] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:54:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:54.431922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de73aa-475f-4c4b-ade5-32805a082d1d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:54.518514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:54.540755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de73aa-475f-4c4b-ade5-32805a082d1d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:54.577832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6d36c-c9fb-46c8-9c05-84f35ed5f75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:54.578502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6d36c-c9fb-46c8-9c05-84f35ed5f75d None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:54.578787 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6d36c-c9fb-46c8-9c05-84f35ed5f75d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:54.579061 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6d36c-c9fb-46c8-9c05-84f35ed5f75d None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:54.579789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6d36c-c9fb-46c8-9c05-84f35ed5f75d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:54.581257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6d36c-c9fb-46c8-9c05-84f35ed5f75d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:54.611116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 28/61] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:54:54 2022] POST /identity/v3/users => generated 275 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:54:57.495812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ccd40-1447-4a14-a337-e581ab70e536 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:57.496823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ccd40-1447-4a14-a337-e581ab70e536 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:57.497309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ccd40-1447-4a14-a337-e581ab70e536 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:57.498941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 34/62] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:57.508762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326642c-634f-4c2b-916a-b4441f10d0e9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:57.509228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326642c-634f-4c2b-916a-b4441f10d0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:57.509326 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326642c-634f-4c2b-916a-b4441f10d0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:57.530712 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a326642c-634f-4c2b-916a-b4441f10d0e9 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:57.532295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a326642c-634f-4c2b-916a-b4441f10d0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:57.610739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 29/63] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:54:57.646721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ccd40-1447-4a14-a337-e581ab70e536 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:57.707170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:57.781366 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ccd40-1447-4a14-a337-e581ab70e536 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:57.828167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91629aeb-67c7-4d90-95d6-13920a319ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:57.828988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91629aeb-67c7-4d90-95d6-13920a319ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:57.829275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91629aeb-67c7-4d90-95d6-13920a319ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:57.829676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91629aeb-67c7-4d90-95d6-13920a319ef6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:57.846032 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91629aeb-67c7-4d90-95d6-13920a319ef6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-91629aeb-67c7-4d90-95d6-13920a319ef6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:54:57.855379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:54:57.856459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:54:57.857406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:54:57.857406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:54:57.857406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:54:57.857406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:54:57.857406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:54:57.857406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:54:57.857406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:54:57.857406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:54:57.857406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:54:57.857406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:54:57.857406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:54:57.857406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:54:57.857406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 35/64] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:54:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:54:57.864661 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326642c-634f-4c2b-916a-b4441f10d0e9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:57.955016 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:58.001580 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326642c-634f-4c2b-916a-b4441f10d0e9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:58.005352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45938f2-0a87-44fb-b70f-bbedf598f696 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.005944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45938f2-0a87-44fb-b70f-bbedf598f696 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.006537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45938f2-0a87-44fb-b70f-bbedf598f696 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.008121 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 36/65] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:54:58.019082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74048c8a-3d30-40c4-b14d-759b47f73217 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.019691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74048c8a-3d30-40c4-b14d-759b47f73217 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.020174 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74048c8a-3d30-40c4-b14d-759b47f73217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.045505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf6200-97fa-4fc5-9af5-8054c101a845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:58.046144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6200-97fa-4fc5-9af5-8054c101a845 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.046430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6200-97fa-4fc5-9af5-8054c101a845 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.046710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6200-97fa-4fc5-9af5-8054c101a845 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.047477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf6200-97fa-4fc5-9af5-8054c101a845 None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:54:58.047791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf6200-97fa-4fc5-9af5-8054c101a845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:58.049392 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf6200-97fa-4fc5-9af5-8054c101a845 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:58.061932 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-74048c8a-3d30-40c4-b14d-759b47f73217 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:58.063163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-74048c8a-3d30-40c4-b14d-759b47f73217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:58.067103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 30/66] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:54:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:58.073412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf6200-97fa-4fc5-9af5-8054c101a845 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:58.172695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 37/67] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:54:58.174694 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976e53a-9e21-4703-9359-c0a20f0bc708 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.175205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976e53a-9e21-4703-9359-c0a20f0bc708 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.175493 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976e53a-9e21-4703-9359-c0a20f0bc708 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.176790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 38/68] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:58.186943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35d46a-4206-4e16-99b8-9c16f4015a9f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.187243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35d46a-4206-4e16-99b8-9c16f4015a9f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.187534 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35d46a-4206-4e16-99b8-9c16f4015a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.209710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0b35d46a-4206-4e16-99b8-9c16f4015a9f None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:58.210849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf6200-97fa-4fc5-9af5-8054c101a845 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:58.211445 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0b35d46a-4206-4e16-99b8-9c16f4015a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:58.239464 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:58.242315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:58.249143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360aaa9-98a8-472a-9c27-24fdeffbdea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:58.250081 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360aaa9-98a8-472a-9c27-24fdeffbdea2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.250364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360aaa9-98a8-472a-9c27-24fdeffbdea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.250649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360aaa9-98a8-472a-9c27-24fdeffbdea2 None admin] PATH_INFO: `/v3/users/fb028e92a3234945a85d5d498019b5ce` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.264679 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360aaa9-98a8-472a-9c27-24fdeffbdea2 None admin] RBAC: Authorizing `identity:get_user(user_id=fb028e92a3234945a85d5d498019b5ce)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:58.266216 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360aaa9-98a8-472a-9c27-24fdeffbdea2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:58.267976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 31/69] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:54:58 2022] GET /identity/v3/users/fb028e92a3234945a85d5d498019b5ce => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:58.268756 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360aaa9-98a8-472a-9c27-24fdeffbdea2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:58.298558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 39/70] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:54:58.298558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f4932-12d3-4bcb-8a51-c6f6351fa929 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.298558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f4932-12d3-4bcb-8a51-c6f6351fa929 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.298558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f4932-12d3-4bcb-8a51-c6f6351fa929 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.298558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 40/71] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:58.298558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f4932-12d3-4bcb-8a51-c6f6351fa929 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:58.382431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360aaa9-98a8-472a-9c27-24fdeffbdea2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:58.412646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f4932-12d3-4bcb-8a51-c6f6351fa929 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:58.417091 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4910b24-aa54-4fdb-9728-5bd0c3815ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:58.417935 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4910b24-aa54-4fdb-9728-5bd0c3815ccc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.418224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4910b24-aa54-4fdb-9728-5bd0c3815ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.418500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4910b24-aa54-4fdb-9728-5bd0c3815ccc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.430330 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4910b24-aa54-4fdb-9728-5bd0c3815ccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b4910b24-aa54-4fdb-9728-5bd0c3815ccc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:54:58.438192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:54:58.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 32/72] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:54:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:54:58.445498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce2b0c-39db-4e2c-b69d-2fee919f2238 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.446227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce2b0c-39db-4e2c-b69d-2fee919f2238 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.446227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce2b0c-39db-4e2c-b69d-2fee919f2238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.456963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66f205-bc66-4d4a-bf50-c1425b05d4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:58.457669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66f205-bc66-4d4a-bf50-c1425b05d4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.457966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66f205-bc66-4d4a-bf50-c1425b05d4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.458242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66f205-bc66-4d4a-bf50-c1425b05d4a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.459193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-62ce2b0c-39db-4e2c-b69d-2fee919f2238 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:58.460343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-62ce2b0c-39db-4e2c-b69d-2fee919f2238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:58.495936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:58.506525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:58.535389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 33/73] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:54:58.536906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce2b0c-39db-4e2c-b69d-2fee919f2238 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:58.550066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66f205-bc66-4d4a-bf50-c1425b05d4a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:58.551869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66f205-bc66-4d4a-bf50-c1425b05d4a3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:58.647451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce2b0c-39db-4e2c-b69d-2fee919f2238 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:58.653510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 41/74] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:54:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:54:58.654169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66f205-bc66-4d4a-bf50-c1425b05d4a3 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:58.681602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2187e-9168-4ce1-809b-cf6b28a35050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:58.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2187e-9168-4ce1-809b-cf6b28a35050 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.682717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2187e-9168-4ce1-809b-cf6b28a35050 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.682998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2187e-9168-4ce1-809b-cf6b28a35050 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.694510 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2187e-9168-4ce1-809b-cf6b28a35050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d6f2187e-9168-4ce1-809b-cf6b28a35050 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:54:58.702583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:54:58.703418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:54:58.704155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:54:58.704155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:54:58.704155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:54:58.704155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:54:58.704155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:54:58.704155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:54:58.704155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:54:58.704155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:54:58.704155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:54:58.704155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:54:58.704155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:54:58.704155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:54:58.704155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 34/75] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:54:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:54:58.706482 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2187e-9168-4ce1-809b-cf6b28a35050 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:58.718686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:58.757637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66f205-bc66-4d4a-bf50-c1425b05d4a3 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:58.762349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:58.795940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd114423-83db-4bfb-b96d-62acf0562e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:58.796647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd114423-83db-4bfb-b96d-62acf0562e5a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.796937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd114423-83db-4bfb-b96d-62acf0562e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.797220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd114423-83db-4bfb-b96d-62acf0562e5a None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.798016 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd114423-83db-4bfb-b96d-62acf0562e5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:54:58.798614 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd114423-83db-4bfb-b96d-62acf0562e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:58.800409 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd114423-83db-4bfb-b96d-62acf0562e5a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:58.809012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 42/76] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:54:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:58.810022 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aa59c-55e3-4745-ad00-db73bc643d45 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.810345 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aa59c-55e3-4745-ad00-db73bc643d45 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.810626 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aa59c-55e3-4745-ad00-db73bc643d45 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.811837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 43/77] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:54:58.812534 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6aa59c-55e3-4745-ad00-db73bc643d45 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:58.834095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2187e-9168-4ce1-809b-cf6b28a35050 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:58.868853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e3301-6727-4aff-812a-9d61361526b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:58.869688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e3301-6727-4aff-812a-9d61361526b6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.869979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e3301-6727-4aff-812a-9d61361526b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.870263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e3301-6727-4aff-812a-9d61361526b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.880760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e3301-6727-4aff-812a-9d61361526b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-081e3301-6727-4aff-812a-9d61361526b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:54:58.888506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:54:58.889534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:54:58.890359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:54:58.890359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:54:58.890359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:54:58.890359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:54:58.890359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:54:58.890359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:54:58.890359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:54:58.890359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:54:58.890359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:54:58.890359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:54:58.890359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:54:58.890359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:54:58.890359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 35/78] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:54:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:54:58.891001 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e3301-6727-4aff-812a-9d61361526b6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:58.919192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6aa59c-55e3-4745-ad00-db73bc643d45 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:58.942746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:58.953808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722725f-e55e-4949-8417-bdfc1b4c0b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:58.954706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722725f-e55e-4949-8417-bdfc1b4c0b80 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:58.954990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722725f-e55e-4949-8417-bdfc1b4c0b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:58.955285 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722725f-e55e-4949-8417-bdfc1b4c0b80 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:58.956522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722725f-e55e-4949-8417-bdfc1b4c0b80 None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:58.958029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722725f-e55e-4949-8417-bdfc1b4c0b80 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:58.959714 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 44/79] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:54:58 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:58.960471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722725f-e55e-4949-8417-bdfc1b4c0b80 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:58.987798 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:58.993911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e3301-6727-4aff-812a-9d61361526b6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:59.028686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222cb46-1952-4998-8419-125bdb6519a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:59.029351 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222cb46-1952-4998-8419-125bdb6519a8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:59.029642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222cb46-1952-4998-8419-125bdb6519a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:59.029944 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222cb46-1952-4998-8419-125bdb6519a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:59.030694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222cb46-1952-4998-8419-125bdb6519a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:54:59.030997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222cb46-1952-4998-8419-125bdb6519a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:59.032414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222cb46-1952-4998-8419-125bdb6519a8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:59.040394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 36/80] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:54:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:59.041563 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aded1-5a85-4c49-93f0-d6daeae1c49f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:59.041892 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aded1-5a85-4c49-93f0-d6daeae1c49f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:59.042169 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aded1-5a85-4c49-93f0-d6daeae1c49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:59.058046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-180aded1-5a85-4c49-93f0-d6daeae1c49f None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:54:59.059164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-180aded1-5a85-4c49-93f0-d6daeae1c49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:54:59.064130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722725f-e55e-4949-8417-bdfc1b4c0b80 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:59.097303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb0423-25d4-4196-a6eb-f26e18d69aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:59.098209 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb0423-25d4-4196-a6eb-f26e18d69aee None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:59.098490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb0423-25d4-4196-a6eb-f26e18d69aee None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:59.098764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb0423-25d4-4196-a6eb-f26e18d69aee None admin] PATH_INFO: `/v3/users/5875b90a72824e1280ee921aaccb7829` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:59.100246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb0423-25d4-4196-a6eb-f26e18d69aee None admin] RBAC: Authorizing `identity:get_user(user_id=5875b90a72824e1280ee921aaccb7829)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:59.102071 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb0423-25d4-4196-a6eb-f26e18d69aee None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:59.103711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 45/81] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:54:58 2022] GET /identity/v3/users/5875b90a72824e1280ee921aaccb7829 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:59.106507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb0423-25d4-4196-a6eb-f26e18d69aee None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:59.125171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 37/82] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:54:59.126066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aded1-5a85-4c49-93f0-d6daeae1c49f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:59.175562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:59.214050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:59.216502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb0423-25d4-4196-a6eb-f26e18d69aee None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:59.227054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aded1-5a85-4c49-93f0-d6daeae1c49f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:59.253189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fb75e-b64c-4055-9b3c-03035c8f3713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:59.253903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fb75e-b64c-4055-9b3c-03035c8f3713 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:59.254188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fb75e-b64c-4055-9b3c-03035c8f3713 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:59.254462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fb75e-b64c-4055-9b3c-03035c8f3713 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:59.255242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fb75e-b64c-4055-9b3c-03035c8f3713 None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:54:59.255557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fb75e-b64c-4055-9b3c-03035c8f3713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:59.256960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fb75e-b64c-4055-9b3c-03035c8f3713 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:59.263652 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e357eaf-dad6-425e-8d55-d0ea2db25956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:59.264317 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e357eaf-dad6-425e-8d55-d0ea2db25956 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:59.264611 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e357eaf-dad6-425e-8d55-d0ea2db25956 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:59.264892 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e357eaf-dad6-425e-8d55-d0ea2db25956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:59.265140 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 46/83] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:54:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:59.265752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e357eaf-dad6-425e-8d55-d0ea2db25956 None admin] RBAC: Adding query filter params (role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d, scope.project.id=27186cff1e94461099e5d6cc062b1abd, user.id=fb028e92a3234945a85d5d498019b5ce) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:54:59.265941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fb75e-b64c-4055-9b3c-03035c8f3713 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:59.266059 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e357eaf-dad6-425e-8d55-d0ea2db25956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:59.267723 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e357eaf-dad6-425e-8d55-d0ea2db25956 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:59.274325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 38/84] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:54:59 2022] GET /identity/v3/role_assignments?user.id=fb028e92a3234945a85d5d498019b5ce&scope.project.id=27186cff1e94461099e5d6cc062b1abd&role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:59.275119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e357eaf-dad6-425e-8d55-d0ea2db25956 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:59.378225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fb75e-b64c-4055-9b3c-03035c8f3713 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:59.404173 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:59.407737 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e357eaf-dad6-425e-8d55-d0ea2db25956 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:59.420120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4241b8-a595-45dd-a6e6-0c23454543f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:59.421211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4241b8-a595-45dd-a6e6-0c23454543f6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:59.421505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4241b8-a595-45dd-a6e6-0c23454543f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:59.421977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4241b8-a595-45dd-a6e6-0c23454543f6 None admin] PATH_INFO: `/v3/users/fb028e92a3234945a85d5d498019b5ce` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:59.423665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4241b8-a595-45dd-a6e6-0c23454543f6 None admin] RBAC: Authorizing `identity:get_user(user_id=fb028e92a3234945a85d5d498019b5ce)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:59.425770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4241b8-a595-45dd-a6e6-0c23454543f6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:59.432640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 47/85] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:54:59 2022] GET /identity/v3/users/fb028e92a3234945a85d5d498019b5ce => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:59.432640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4241b8-a595-45dd-a6e6-0c23454543f6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:59.447037 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f033e1e-eb90-4b40-b461-5c30205f53f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:59.447713 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f033e1e-eb90-4b40-b461-5c30205f53f9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:59.448002 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f033e1e-eb90-4b40-b461-5c30205f53f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:59.448282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f033e1e-eb90-4b40-b461-5c30205f53f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:59.469692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:59.523949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f033e1e-eb90-4b40-b461-5c30205f53f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:59.525444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f033e1e-eb90-4b40-b461-5c30205f53f9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:59.557567 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4241b8-a595-45dd-a6e6-0c23454543f6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:59.592177 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02f47d-8e70-4d67-b036-8bd1779a6202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:59.593104 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02f47d-8e70-4d67-b036-8bd1779a6202 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:59.593393 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02f47d-8e70-4d67-b036-8bd1779a6202 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:59.593698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02f47d-8e70-4d67-b036-8bd1779a6202 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:59.594944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02f47d-8e70-4d67-b036-8bd1779a6202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:59.596500 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02f47d-8e70-4d67-b036-8bd1779a6202 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:59.603089 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 48/86] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:54:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:59.603089 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02f47d-8e70-4d67-b036-8bd1779a6202 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:59.630531 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 39/87] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:54:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:54:59.635859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f033e1e-eb90-4b40-b461-5c30205f53f9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:59.662815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:59.693352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:59.707591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02f47d-8e70-4d67-b036-8bd1779a6202 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:59.742224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b9f74-835b-46ee-835e-69b283addff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:59.743126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b9f74-835b-46ee-835e-69b283addff4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:59.743411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b9f74-835b-46ee-835e-69b283addff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:59.743688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b9f74-835b-46ee-835e-69b283addff4 None admin] PATH_INFO: `/v3/users/fb028e92a3234945a85d5d498019b5ce` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:59.744624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f033e1e-eb90-4b40-b461-5c30205f53f9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:59.744878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b9f74-835b-46ee-835e-69b283addff4 None admin] RBAC: Authorizing `identity:get_user(user_id=fb028e92a3234945a85d5d498019b5ce)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:59.746475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b9f74-835b-46ee-835e-69b283addff4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:59.752770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 49/88] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:54:59 2022] GET /identity/v3/users/fb028e92a3234945a85d5d498019b5ce => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:54:59.752770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b9f74-835b-46ee-835e-69b283addff4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:54:59.786813 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075e943-59fc-4d2e-83de-e6a0b7bd391b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:59.787775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075e943-59fc-4d2e-83de-e6a0b7bd391b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:59.788228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075e943-59fc-4d2e-83de-e6a0b7bd391b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:59.788660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075e943-59fc-4d2e-83de-e6a0b7bd391b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:59.853463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b9f74-835b-46ee-835e-69b283addff4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:54:59.865578 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075e943-59fc-4d2e-83de-e6a0b7bd391b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:59.867116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075e943-59fc-4d2e-83de-e6a0b7bd391b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:59.883559 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:59.893847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28afe54-a77f-4607-89c4-1684ff32087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:54:59.894547 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28afe54-a77f-4607-89c4-1684ff32087f None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:54:59.894884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28afe54-a77f-4607-89c4-1684ff32087f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:54:59.895294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28afe54-a77f-4607-89c4-1684ff32087f None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:54:59.895928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28afe54-a77f-4607-89c4-1684ff32087f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:54:59.897456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28afe54-a77f-4607-89c4-1684ff32087f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:54:59.923994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:54:59.933657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 50/89] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:54:59 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:54:59.933657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28afe54-a77f-4607-89c4-1684ff32087f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:00.006430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 40/90] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:54:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:00.007456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075e943-59fc-4d2e-83de-e6a0b7bd391b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:00.069915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28afe54-a77f-4607-89c4-1684ff32087f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:00.115928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f48d4-1533-4958-91c9-84064e9d921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:00.117169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f48d4-1533-4958-91c9-84064e9d921f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:00.117716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f48d4-1533-4958-91c9-84064e9d921f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:00.118628 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f48d4-1533-4958-91c9-84064e9d921f None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:00.122858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f48d4-1533-4958-91c9-84064e9d921f None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:00.126414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f48d4-1533-4958-91c9-84064e9d921f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:00.128965 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 51/91] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:54:59 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:00.129814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f48d4-1533-4958-91c9-84064e9d921f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:00.131215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075e943-59fc-4d2e-83de-e6a0b7bd391b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:00.184877 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e946751-e0c9-4485-bedd-5a2554b20898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:00.185546 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e946751-e0c9-4485-bedd-5a2554b20898 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:00.185866 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e946751-e0c9-4485-bedd-5a2554b20898 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:00.186149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e946751-e0c9-4485-bedd-5a2554b20898 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:00.196408 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:00.217445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:00.259849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f48d4-1533-4958-91c9-84064e9d921f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:00.280450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e946751-e0c9-4485-bedd-5a2554b20898 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:00.282131 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e946751-e0c9-4485-bedd-5a2554b20898 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:00.303909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c3d3b-5a87-41a2-a696-d93e1b290c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:00.304782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c3d3b-5a87-41a2-a696-d93e1b290c86 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:00.305077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c3d3b-5a87-41a2-a696-d93e1b290c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:00.305358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c3d3b-5a87-41a2-a696-d93e1b290c86 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:00.306702 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c3d3b-5a87-41a2-a696-d93e1b290c86 None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:00.308162 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c3d3b-5a87-41a2-a696-d93e1b290c86 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:00.310009 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 52/92] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:00 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:00.311303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c3d3b-5a87-41a2-a696-d93e1b290c86 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:00.420579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 41/93] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:00.421401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e946751-e0c9-4485-bedd-5a2554b20898 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:00.436438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c3d3b-5a87-41a2-a696-d93e1b290c86 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:00.460264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:00.483278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19814b-71d3-4b0c-8988-9dff4f96294a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:00.483953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19814b-71d3-4b0c-8988-9dff4f96294a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:00.484236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19814b-71d3-4b0c-8988-9dff4f96294a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:00.484513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19814b-71d3-4b0c-8988-9dff4f96294a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:00.485358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19814b-71d3-4b0c-8988-9dff4f96294a None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=27186cff1e94461099e5d6cc062b1abd, user.id=5875b90a72824e1280ee921aaccb7829) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:00.485724 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19814b-71d3-4b0c-8988-9dff4f96294a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:00.487206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19814b-71d3-4b0c-8988-9dff4f96294a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:00.492593 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:00.500798 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 53/94] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:00 2022] GET /identity/v3/role_assignments?user.id=5875b90a72824e1280ee921aaccb7829&scope.project.id=27186cff1e94461099e5d6cc062b1abd&role.id=a3de9df253294d069dc1867fa26600c3 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:00.501862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19814b-71d3-4b0c-8988-9dff4f96294a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:00.584266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e946751-e0c9-4485-bedd-5a2554b20898 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:00.671833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d0d76-ddf9-4303-b535-acfa667c4434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:00.672469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d0d76-ddf9-4303-b535-acfa667c4434 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:00.672750 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d0d76-ddf9-4303-b535-acfa667c4434 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:00.673030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d0d76-ddf9-4303-b535-acfa667c4434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:00.677503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d0d76-ddf9-4303-b535-acfa667c4434 None admin] RBAC: Adding query filter params (role.id=78a15fc7951d4f28beb57f54dace457d, scope.project.id=27186cff1e94461099e5d6cc062b1abd, user.id=fb028e92a3234945a85d5d498019b5ce) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:00.677503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d0d76-ddf9-4303-b535-acfa667c4434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:00.677503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d0d76-ddf9-4303-b535-acfa667c4434 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:00.686317 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 42/95] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:00 2022] GET /identity/v3/role_assignments?user.id=fb028e92a3234945a85d5d498019b5ce&scope.project.id=27186cff1e94461099e5d6cc062b1abd&role.id=78a15fc7951d4f28beb57f54dace457d => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:00.749359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19814b-71d3-4b0c-8988-9dff4f96294a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:00.795196 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345d590-c98e-4e23-a326-75552ecd33ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:00.796477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345d590-c98e-4e23-a326-75552ecd33ad None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:00.797067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345d590-c98e-4e23-a326-75552ecd33ad None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:00.797548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345d590-c98e-4e23-a326-75552ecd33ad None admin] PATH_INFO: `/v3/projects/361fcd58851a4377a4e4d372ef8c8c83` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:00.799490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345d590-c98e-4e23-a326-75552ecd33ad None admin] RBAC: Authorizing `identity:get_project(project_id=361fcd58851a4377a4e4d372ef8c8c83)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:00.801407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345d590-c98e-4e23-a326-75552ecd33ad None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:00.803814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 54/96] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:00 2022] GET /identity/v3/projects/361fcd58851a4377a4e4d372ef8c8c83 => generated 312 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:00.809840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d0d76-ddf9-4303-b535-acfa667c4434 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:00.942180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d0d76-ddf9-4303-b535-acfa667c4434 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:00.963711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:00.991526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba29c3-1977-4b7b-988e-121bc2675bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:00.992576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba29c3-1977-4b7b-988e-121bc2675bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:00.993058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba29c3-1977-4b7b-988e-121bc2675bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:00.993551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba29c3-1977-4b7b-988e-121bc2675bb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:00.994651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba29c3-1977-4b7b-988e-121bc2675bb2 None admin] RBAC: Adding query filter params (role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d, scope.project.id=361fcd58851a4377a4e4d372ef8c8c83, user.id=fb028e92a3234945a85d5d498019b5ce) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:00.995166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba29c3-1977-4b7b-988e-121bc2675bb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:00.996845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba29c3-1977-4b7b-988e-121bc2675bb2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:01.004240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 43/97] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:00 2022] GET /identity/v3/role_assignments?user.id=fb028e92a3234945a85d5d498019b5ce&scope.project.id=361fcd58851a4377a4e4d372ef8c8c83&role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:01.544210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f164b-d330-4618-8383-a17b78737b2b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:01.544632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f164b-d330-4618-8383-a17b78737b2b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:01.544808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f164b-d330-4618-8383-a17b78737b2b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:01.546226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 55/98] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:01.551766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7a05c-7e5b-4305-b99d-a21bd5f0b413 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:01.552067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7a05c-7e5b-4305-b99d-a21bd5f0b413 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:01.552352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7a05c-7e5b-4305-b99d-a21bd5f0b413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:01.574638 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-fdf7a05c-7e5b-4305-b99d-a21bd5f0b413 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:01.575863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fdf7a05c-7e5b-4305-b99d-a21bd5f0b413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:01.664026 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 44/99] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:01.698691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f164b-d330-4618-8383-a17b78737b2b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:01.776497 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:01.841501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f164b-d330-4618-8383-a17b78737b2b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:01.886865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d5d51-0b97-44e7-a60a-ab898adbaea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:01.887688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d5d51-0b97-44e7-a60a-ab898adbaea7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:01.887980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d5d51-0b97-44e7-a60a-ab898adbaea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:01.888259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d5d51-0b97-44e7-a60a-ab898adbaea7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:01.903928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d5d51-0b97-44e7-a60a-ab898adbaea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-233d5d51-0b97-44e7-a60a-ab898adbaea7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:01.912207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:01.913050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:01.914462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:01.914462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:01.914462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:01.914462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:01.914462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:01.914462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:01.914462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:01.914462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:01.914462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:01.914462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:01.914462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:01.914462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:01.914462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 56/100] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:01.919731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7a05c-7e5b-4305-b99d-a21bd5f0b413 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:02.045441 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7a05c-7e5b-4305-b99d-a21bd5f0b413 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:02.056453 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:02.087445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db24c3-023d-4147-9ae0-02fe0e18fa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:02.088129 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db24c3-023d-4147-9ae0-02fe0e18fa45 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:02.088417 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db24c3-023d-4147-9ae0-02fe0e18fa45 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:02.088698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db24c3-023d-4147-9ae0-02fe0e18fa45 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:02.089463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db24c3-023d-4147-9ae0-02fe0e18fa45 None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:02.089799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db24c3-023d-4147-9ae0-02fe0e18fa45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:02.091220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db24c3-023d-4147-9ae0-02fe0e18fa45 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:02.099966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 45/101] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:02.106372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d5d51-0b97-44e7-a60a-ab898adbaea7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:02.258407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d5d51-0b97-44e7-a60a-ab898adbaea7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:02.277638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:02.307230 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76580b30-f679-4366-a650-79fec451682a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:02.308484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76580b30-f679-4366-a650-79fec451682a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:02.308974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76580b30-f679-4366-a650-79fec451682a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:02.309464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76580b30-f679-4366-a650-79fec451682a None admin] PATH_INFO: `/v3/users/fb028e92a3234945a85d5d498019b5ce` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:02.311015 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76580b30-f679-4366-a650-79fec451682a None admin] RBAC: Authorizing `identity:get_user(user_id=fb028e92a3234945a85d5d498019b5ce)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:02.312769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76580b30-f679-4366-a650-79fec451682a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:02.315368 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 57/102] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:02 2022] GET /identity/v3/users/fb028e92a3234945a85d5d498019b5ce => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:02.321340 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db24c3-023d-4147-9ae0-02fe0e18fa45 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:02.469512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db24c3-023d-4147-9ae0-02fe0e18fa45 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:02.505728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e8f4d-c1b6-4537-a67b-faf1c24ec8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:02.506685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e8f4d-c1b6-4537-a67b-faf1c24ec8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:02.506977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e8f4d-c1b6-4537-a67b-faf1c24ec8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:02.507264 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e8f4d-c1b6-4537-a67b-faf1c24ec8b7 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:02.508595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e8f4d-c1b6-4537-a67b-faf1c24ec8b7 None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:02.510178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e8f4d-c1b6-4537-a67b-faf1c24ec8b7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:02.512000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 46/103] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:02 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:02.518442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76580b30-f679-4366-a650-79fec451682a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:02.682500 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76580b30-f679-4366-a650-79fec451682a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:02.734029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987f817-ee5a-4e42-8eb5-7278c8e3b6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:02.735191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987f817-ee5a-4e42-8eb5-7278c8e3b6e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:02.735657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987f817-ee5a-4e42-8eb5-7278c8e3b6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:02.736258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987f817-ee5a-4e42-8eb5-7278c8e3b6e9 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd/users/fb028e92a3234945a85d5d498019b5ce/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:02.742941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987f817-ee5a-4e42-8eb5-7278c8e3b6e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=27186cff1e94461099e5d6cc062b1abd, user_id=fb028e92a3234945a85d5d498019b5ce, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:02.744557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987f817-ee5a-4e42-8eb5-7278c8e3b6e9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:02.776462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 58/104] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:55:02 2022] PUT /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd/users/fb028e92a3234945a85d5d498019b5ce/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:55:02.847285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b4578-64d4-48f9-b369-b8ae6ca957e2 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:02.847793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b4578-64d4-48f9-b369-b8ae6ca957e2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:02.848293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b4578-64d4-48f9-b369-b8ae6ca957e2 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:02.850024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 47/105] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:02.855772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6891e7-3e7c-4e56-832b-4843c22f8f13 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:02.856598 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6891e7-3e7c-4e56-832b-4843c22f8f13 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:02.857024 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6891e7-3e7c-4e56-832b-4843c22f8f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:02.886693 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-bd6891e7-3e7c-4e56-832b-4843c22f8f13 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:02.888152 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bd6891e7-3e7c-4e56-832b-4843c22f8f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:02.928045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:02.984418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 59/106] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:03.025888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b4578-64d4-48f9-b369-b8ae6ca957e2 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:03.083796 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:03.168253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b4578-64d4-48f9-b369-b8ae6ca957e2 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:03.206136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2fdd6-9e3d-4821-96ef-27083a0d6008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:03.206957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2fdd6-9e3d-4821-96ef-27083a0d6008 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:03.207303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2fdd6-9e3d-4821-96ef-27083a0d6008 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:03.207589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2fdd6-9e3d-4821-96ef-27083a0d6008 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:03.219537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2fdd6-9e3d-4821-96ef-27083a0d6008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-0ac2fdd6-9e3d-4821-96ef-27083a0d6008 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:03.228323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:03.229182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:03.229984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:03.229984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:03.229984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:03.229984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:03.229984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:03.229984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:03.229984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:03.229984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:03.229984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:03.229984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:03.229984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:03.229984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:03.229984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 48/107] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:03.235632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6891e7-3e7c-4e56-832b-4843c22f8f13 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:03.444638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6891e7-3e7c-4e56-832b-4843c22f8f13 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:03.514455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2d5fe-9201-4028-bc94-2c956575443e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:03.515591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2d5fe-9201-4028-bc94-2c956575443e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:03.516077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2d5fe-9201-4028-bc94-2c956575443e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:03.516567 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2d5fe-9201-4028-bc94-2c956575443e None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:03.518066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2d5fe-9201-4028-bc94-2c956575443e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:03.518602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2d5fe-9201-4028-bc94-2c956575443e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:03.520229 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2d5fe-9201-4028-bc94-2c956575443e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:03.534560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 60/108] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:03.540373 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2fdd6-9e3d-4821-96ef-27083a0d6008 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:03.771606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728873e9-4a5c-4c94-8df0-8bd04a5e7a67 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:03.771789 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728873e9-4a5c-4c94-8df0-8bd04a5e7a67 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:03.772083 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728873e9-4a5c-4c94-8df0-8bd04a5e7a67 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:03.773271 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:03.773465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 61/109] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:03.777847 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2fdd6-9e3d-4821-96ef-27083a0d6008 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:03.785858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0515c-2275-43aa-b19b-94fc46c73ed2 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:03.786380 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0515c-2275-43aa-b19b-94fc46c73ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:03.786802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0515c-2275-43aa-b19b-94fc46c73ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:03.826921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9ec0515c-2275-43aa-b19b-94fc46c73ed2 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:03.828551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9ec0515c-2275-43aa-b19b-94fc46c73ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:03.866239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:03.866841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b76437-bf8c-43fc-9aba-922be073455c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:03.868500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b76437-bf8c-43fc-9aba-922be073455c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:03.868971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b76437-bf8c-43fc-9aba-922be073455c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:03.869401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b76437-bf8c-43fc-9aba-922be073455c None admin] PATH_INFO: `/v3/users/dde6b24774644a72ae522439f0e1e857` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:03.892923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b76437-bf8c-43fc-9aba-922be073455c None admin] RBAC: Authorizing `identity:get_user(user_id=dde6b24774644a72ae522439f0e1e857)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:03.894926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b76437-bf8c-43fc-9aba-922be073455c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:03.896867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 49/110] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:03 2022] GET /identity/v3/users/dde6b24774644a72ae522439f0e1e857 => generated 289 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:03.900186 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6a572-9eb6-4065-a896-306a4f193ff9 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:03.902603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6a572-9eb6-4065-a896-306a4f193ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:03.903071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6a572-9eb6-4065-a896-306a4f193ff9 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:03.906360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 50/111] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:03.909181 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6a572-9eb6-4065-a896-306a4f193ff9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:03.991153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 62/112] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:03.992843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07d073-1629-4f87-accf-0d319ae78b18 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:03.993335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07d073-1629-4f87-accf-0d319ae78b18 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:03.993854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07d073-1629-4f87-accf-0d319ae78b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:04.027944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9c07d073-1629-4f87-accf-0d319ae78b18 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:04.029510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9c07d073-1629-4f87-accf-0d319ae78b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:04.135767 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6a572-9eb6-4065-a896-306a4f193ff9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:04.136855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 63/113] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:04.137921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07d073-1629-4f87-accf-0d319ae78b18 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:04.185384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba089637-787e-41d8-9e54-d6b585f82887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:04.186746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba089637-787e-41d8-9e54-d6b585f82887 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:04.187297 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba089637-787e-41d8-9e54-d6b585f82887 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:04.187803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba089637-787e-41d8-9e54-d6b585f82887 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:04.218093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:04.252889 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:04.263396 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07d073-1629-4f87-accf-0d319ae78b18 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:04.264434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba089637-787e-41d8-9e54-d6b585f82887 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:04.267281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba089637-787e-41d8-9e54-d6b585f82887 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:04.301558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dea9a7-0fea-4323-9aed-2708d8a4c21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:04.302382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dea9a7-0fea-4323-9aed-2708d8a4c21e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:04.302666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dea9a7-0fea-4323-9aed-2708d8a4c21e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:04.302945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dea9a7-0fea-4323-9aed-2708d8a4c21e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:04.304147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dea9a7-0fea-4323-9aed-2708d8a4c21e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:04.305635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dea9a7-0fea-4323-9aed-2708d8a4c21e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:04.311964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 64/114] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:04.311964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e08ae-b8dd-4846-adf5-fa113bba2450 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:04.311964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e08ae-b8dd-4846-adf5-fa113bba2450 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:04.311964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e08ae-b8dd-4846-adf5-fa113bba2450 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:04.311964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 65/115] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:04.311964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e08ae-b8dd-4846-adf5-fa113bba2450 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:04.378407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 51/116] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:04.379765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5bbb-0afd-4e4c-ba7b-620c510b692b None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:04.380058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5bbb-0afd-4e4c-ba7b-620c510b692b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:04.380335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5bbb-0afd-4e4c-ba7b-620c510b692b None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:04.381795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 52/117] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:04.382291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d5bbb-0afd-4e4c-ba7b-620c510b692b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:04.424534 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e08ae-b8dd-4846-adf5-fa113bba2450 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:04.447155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:04.461166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28d60c-983f-4f14-a136-5d959cb6aec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:04.462041 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28d60c-983f-4f14-a136-5d959cb6aec7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:04.462332 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28d60c-983f-4f14-a136-5d959cb6aec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:04.462632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28d60c-983f-4f14-a136-5d959cb6aec7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:04.474027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28d60c-983f-4f14-a136-5d959cb6aec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-4d28d60c-983f-4f14-a136-5d959cb6aec7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:04.482524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:04.483348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:04.484080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:04.484080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:04.484080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:04.484080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:04.484080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:04.484080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:04.484080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:04.484080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:04.484080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:04.484080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:04.484080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:04.484080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:04.484080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 66/118] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:55:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:04.487791 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d5532-c521-416c-8ef2-655a5efcad00 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:04.488522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d5532-c521-416c-8ef2-655a5efcad00 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:04.489011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d5532-c521-416c-8ef2-655a5efcad00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:04.496215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d5bbb-0afd-4e4c-ba7b-620c510b692b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:04.506950 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-bd7d5532-c521-416c-8ef2-655a5efcad00 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:04.517073 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bd7d5532-c521-416c-8ef2-655a5efcad00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:04.531266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5db86-b325-4c9e-8dd7-61e5d9f0fb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:04.532027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5db86-b325-4c9e-8dd7-61e5d9f0fb70 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:04.532369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5db86-b325-4c9e-8dd7-61e5d9f0fb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:04.532768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5db86-b325-4c9e-8dd7-61e5d9f0fb70 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:04.533666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5db86-b325-4c9e-8dd7-61e5d9f0fb70 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:04.535307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5db86-b325-4c9e-8dd7-61e5d9f0fb70 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:04.542920 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:04.566297 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 53/119] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:55:04 2022] POST /identity/v3/projects => generated 302 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:55:04.567019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5db86-b325-4c9e-8dd7-61e5d9f0fb70 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:04.620717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 67/120] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:04.622650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fbd8a-bee5-4f97-bf9d-1f619cbc61b3 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:04.622650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fbd8a-bee5-4f97-bf9d-1f619cbc61b3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:04.622949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fbd8a-bee5-4f97-bf9d-1f619cbc61b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:04.641328 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-349fbd8a-bee5-4f97-bf9d-1f619cbc61b3 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:04.643243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-349fbd8a-bee5-4f97-bf9d-1f619cbc61b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:04.694513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5db86-b325-4c9e-8dd7-61e5d9f0fb70 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:04.707830 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:04.723093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 68/121] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:04.723987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fbd8a-bee5-4f97-bf9d-1f619cbc61b3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:04.737492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70fba5-eac6-4446-838c-3cf4451168c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:04.738784 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70fba5-eac6-4446-838c-3cf4451168c0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:04.739311 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70fba5-eac6-4446-838c-3cf4451168c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:04.739786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70fba5-eac6-4446-838c-3cf4451168c0 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:04.741293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70fba5-eac6-4446-838c-3cf4451168c0 None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:04.743248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70fba5-eac6-4446-838c-3cf4451168c0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:04.745252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 54/122] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:04 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:04.746261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70fba5-eac6-4446-838c-3cf4451168c0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:04.819826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:04.850381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fbd8a-bee5-4f97-bf9d-1f619cbc61b3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:04.862481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70fba5-eac6-4446-838c-3cf4451168c0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:04.887841 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c765c7-2c5c-4c36-8667-ccc7b90156f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:04.888623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c765c7-2c5c-4c36-8667-ccc7b90156f1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:04.889009 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c765c7-2c5c-4c36-8667-ccc7b90156f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:04.889297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c765c7-2c5c-4c36-8667-ccc7b90156f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:04.890253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c765c7-2c5c-4c36-8667-ccc7b90156f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:04.890570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c765c7-2c5c-4c36-8667-ccc7b90156f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:04.892215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c765c7-2c5c-4c36-8667-ccc7b90156f1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:04.898543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cc7a0-5642-4531-b4f8-6a36f050cc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:04.899355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cc7a0-5642-4531-b4f8-6a36f050cc23 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:04.899642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cc7a0-5642-4531-b4f8-6a36f050cc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:04.899969 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cc7a0-5642-4531-b4f8-6a36f050cc23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:04.903287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 69/123] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:04.904125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c765c7-2c5c-4c36-8667-ccc7b90156f1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:04.911983 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cc7a0-5642-4531-b4f8-6a36f050cc23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-a62cc7a0-5642-4531-b4f8-6a36f050cc23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:04.919851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:04.920674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:04.921524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:04.921524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:04.921524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:04.921524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:04.921524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:04.921524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:04.921524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:04.921524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:04.921524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:04.921524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:04.921524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:04.921524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:04.921524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 55/124] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:04.922174 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cc7a0-5642-4531-b4f8-6a36f050cc23 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:04.962847 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:05.014260 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c765c7-2c5c-4c36-8667-ccc7b90156f1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:05.025516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cc7a0-5642-4531-b4f8-6a36f050cc23 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:05.051612 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6c4ae-7841-486e-9367-7510331284dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:05.052383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6c4ae-7841-486e-9367-7510331284dd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:05.052680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6c4ae-7841-486e-9367-7510331284dd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:05.052961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6c4ae-7841-486e-9367-7510331284dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:05.053825 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6c4ae-7841-486e-9367-7510331284dd None admin] RBAC: Adding query filter params (role.id=f420d4b9bfb64b0f8cd462e9d72bd24d, scope.project.id=27186cff1e94461099e5d6cc062b1abd, user.id=dde6b24774644a72ae522439f0e1e857) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:05.054141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6c4ae-7841-486e-9367-7510331284dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:05.055815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6c4ae-7841-486e-9367-7510331284dd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:05.059909 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ebb8b-fd3f-4103-b943-f586896ea7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:05.060718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ebb8b-fd3f-4103-b943-f586896ea7bf None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:05.061002 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ebb8b-fd3f-4103-b943-f586896ea7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:05.061282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ebb8b-fd3f-4103-b943-f586896ea7bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:05.062469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 70/125] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:04 2022] GET /identity/v3/role_assignments?user.id=dde6b24774644a72ae522439f0e1e857&scope.project.id=27186cff1e94461099e5d6cc062b1abd&role.id=f420d4b9bfb64b0f8cd462e9d72bd24d => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:05.063383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6c4ae-7841-486e-9367-7510331284dd None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:05.067228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:05.074970 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ebb8b-fd3f-4103-b943-f586896ea7bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8b2ebb8b-fd3f-4103-b943-f586896ea7bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:05.083646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:05.085714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 56/126] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:55:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:05.087196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ebb8b-fd3f-4103-b943-f586896ea7bf None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:05.274359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6c4ae-7841-486e-9367-7510331284dd None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:05.308881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ebb8b-fd3f-4103-b943-f586896ea7bf None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:05.326035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:05.331118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f43f93-8fb8-4c2b-8971-eaccff5feadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:05.332017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f43f93-8fb8-4c2b-8971-eaccff5feadf None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:05.332352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f43f93-8fb8-4c2b-8971-eaccff5feadf None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:05.332700 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f43f93-8fb8-4c2b-8971-eaccff5feadf None admin] PATH_INFO: `/v3/users/fb028e92a3234945a85d5d498019b5ce` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:05.334738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f43f93-8fb8-4c2b-8971-eaccff5feadf None admin] RBAC: Authorizing `identity:get_user(user_id=fb028e92a3234945a85d5d498019b5ce)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:05.336227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f43f93-8fb8-4c2b-8971-eaccff5feadf None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:05.337915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 71/127] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:05 2022] GET /identity/v3/users/fb028e92a3234945a85d5d498019b5ce => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:05.340314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f43f93-8fb8-4c2b-8971-eaccff5feadf None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:05.372125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0cfe0-46e8-4470-a9e7-15091e6ae53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:05.372806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0cfe0-46e8-4470-a9e7-15091e6ae53f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:05.373090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0cfe0-46e8-4470-a9e7-15091e6ae53f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:05.373370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0cfe0-46e8-4470-a9e7-15091e6ae53f None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:05.374338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0cfe0-46e8-4470-a9e7-15091e6ae53f None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:05.374648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0cfe0-46e8-4470-a9e7-15091e6ae53f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:05.376058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0cfe0-46e8-4470-a9e7-15091e6ae53f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:05.390169 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 57/128] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:05.392453 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0cfe0-46e8-4470-a9e7-15091e6ae53f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:05.476695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:05.528254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f43f93-8fb8-4c2b-8971-eaccff5feadf None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:05.563414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0cfe0-46e8-4470-a9e7-15091e6ae53f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:05.575547 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca189f9-f4b8-4ea3-b7e3-0955949f91cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:05.576814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca189f9-f4b8-4ea3-b7e3-0955949f91cc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:05.577050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca189f9-f4b8-4ea3-b7e3-0955949f91cc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:05.577338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca189f9-f4b8-4ea3-b7e3-0955949f91cc None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:05.578333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca189f9-f4b8-4ea3-b7e3-0955949f91cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:05.579040 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca189f9-f4b8-4ea3-b7e3-0955949f91cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:05.580847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca189f9-f4b8-4ea3-b7e3-0955949f91cc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:05.592382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 72/129] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:05.593239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca189f9-f4b8-4ea3-b7e3-0955949f91cc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:05.608318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ee054-fd04-41b3-8b01-cbd8bdcb90ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:05.609251 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ee054-fd04-41b3-8b01-cbd8bdcb90ef None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:05.609541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ee054-fd04-41b3-8b01-cbd8bdcb90ef None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:05.609852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ee054-fd04-41b3-8b01-cbd8bdcb90ef None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:05.611198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ee054-fd04-41b3-8b01-cbd8bdcb90ef None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:05.612918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ee054-fd04-41b3-8b01-cbd8bdcb90ef None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:05.615207 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 58/130] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:05 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:05.616503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb717ad-cd9e-4303-b2f6-ff4fd8e94bde None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:05.616796 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb717ad-cd9e-4303-b2f6-ff4fd8e94bde None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:05.617127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb717ad-cd9e-4303-b2f6-ff4fd8e94bde None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:05.622719 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 59/131] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:05.623569 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb717ad-cd9e-4303-b2f6-ff4fd8e94bde None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:05.663651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:05.726324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca189f9-f4b8-4ea3-b7e3-0955949f91cc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:05.755355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb717ad-cd9e-4303-b2f6-ff4fd8e94bde None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:05.770828 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f033730-25e0-441a-9f44-bf62691c88a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:05.772437 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f033730-25e0-441a-9f44-bf62691c88a9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:05.773067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f033730-25e0-441a-9f44-bf62691c88a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:05.773597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f033730-25e0-441a-9f44-bf62691c88a9 None admin] PATH_INFO: `/v3/users/fb028e92a3234945a85d5d498019b5ce` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:05.775189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f033730-25e0-441a-9f44-bf62691c88a9 None admin] RBAC: Authorizing `identity:get_user(user_id=fb028e92a3234945a85d5d498019b5ce)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:05.777143 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f033730-25e0-441a-9f44-bf62691c88a9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:05.779235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 73/132] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:05 2022] GET /identity/v3/users/fb028e92a3234945a85d5d498019b5ce => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:05.780624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f033730-25e0-441a-9f44-bf62691c88a9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:05.785342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:05.801649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262fcad-5788-4f99-acc2-8fa3c25c2ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:05.803166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262fcad-5788-4f99-acc2-8fa3c25c2ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:05.803570 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262fcad-5788-4f99-acc2-8fa3c25c2ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:05.804506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262fcad-5788-4f99-acc2-8fa3c25c2ed0 None admin] PATH_INFO: `/v3/users/5875b90a72824e1280ee921aaccb7829` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:05.805976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262fcad-5788-4f99-acc2-8fa3c25c2ed0 None admin] RBAC: Authorizing `identity:get_user(user_id=5875b90a72824e1280ee921aaccb7829)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:05.807787 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262fcad-5788-4f99-acc2-8fa3c25c2ed0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:05.810227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 60/133] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:05 2022] GET /identity/v3/users/5875b90a72824e1280ee921aaccb7829 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:05.816282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f280645-a8ec-44cf-918c-7cab11ba8598 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:05.816905 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f280645-a8ec-44cf-918c-7cab11ba8598 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:05.817136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f280645-a8ec-44cf-918c-7cab11ba8598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:05.839685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1f280645-a8ec-44cf-918c-7cab11ba8598 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:05.841370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1f280645-a8ec-44cf-918c-7cab11ba8598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:05.920180 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f033730-25e0-441a-9f44-bf62691c88a9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:05.934137 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 61/134] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:05.935237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f280645-a8ec-44cf-918c-7cab11ba8598 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:05.970327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:05.990531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687011fb-e118-4442-9aff-136ff3cd063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:05.992116 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687011fb-e118-4442-9aff-136ff3cd063c None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:05.992555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687011fb-e118-4442-9aff-136ff3cd063c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:05.992977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687011fb-e118-4442-9aff-136ff3cd063c None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd/users/fb028e92a3234945a85d5d498019b5ce/roles/78a15fc7951d4f28beb57f54dace457d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:05.996263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687011fb-e118-4442-9aff-136ff3cd063c None admin] RBAC: Authorizing `identity:create_grant(project_id=27186cff1e94461099e5d6cc062b1abd, user_id=fb028e92a3234945a85d5d498019b5ce, role_id=78a15fc7951d4f28beb57f54dace457d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:05.998601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687011fb-e118-4442-9aff-136ff3cd063c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:06.020793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 74/135] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:55:05 2022] PUT /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd/users/fb028e92a3234945a85d5d498019b5ce/roles/78a15fc7951d4f28beb57f54dace457d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:55:06.022472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687011fb-e118-4442-9aff-136ff3cd063c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:06.080598 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f280645-a8ec-44cf-918c-7cab11ba8598 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:06.119957 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:06.124883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041ca7a-3e8b-41eb-963b-5282d130ccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:06.126017 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041ca7a-3e8b-41eb-963b-5282d130ccd7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:06.126429 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041ca7a-3e8b-41eb-963b-5282d130ccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:06.126803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041ca7a-3e8b-41eb-963b-5282d130ccd7 None admin] PATH_INFO: `/v3/projects/361fcd58851a4377a4e4d372ef8c8c83` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:06.129949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041ca7a-3e8b-41eb-963b-5282d130ccd7 None admin] RBAC: Authorizing `identity:get_project(project_id=361fcd58851a4377a4e4d372ef8c8c83)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:06.131843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041ca7a-3e8b-41eb-963b-5282d130ccd7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:06.135637 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 62/136] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:05 2022] GET /identity/v3/projects/361fcd58851a4377a4e4d372ef8c8c83 => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:06.135938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041ca7a-3e8b-41eb-963b-5282d130ccd7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:06.178270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687011fb-e118-4442-9aff-136ff3cd063c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:06.219941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d6777-4813-4772-b3bf-38df7e0225c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:06.221587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d6777-4813-4772-b3bf-38df7e0225c8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:06.222288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d6777-4813-4772-b3bf-38df7e0225c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:06.222884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d6777-4813-4772-b3bf-38df7e0225c8 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:06.225002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d6777-4813-4772-b3bf-38df7e0225c8 None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:06.227378 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d6777-4813-4772-b3bf-38df7e0225c8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:06.229510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 75/137] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:06 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:06.234528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d6777-4813-4772-b3bf-38df7e0225c8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:06.266318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:06.289306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041ca7a-3e8b-41eb-963b-5282d130ccd7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:06.350033 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3004c7-8243-4751-a9e4-643ed6296326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:06.351285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3004c7-8243-4751-a9e4-643ed6296326 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:06.351576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3004c7-8243-4751-a9e4-643ed6296326 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:06.353332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3004c7-8243-4751-a9e4-643ed6296326 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:06.371002 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3004c7-8243-4751-a9e4-643ed6296326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-ef3004c7-8243-4751-a9e4-643ed6296326 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:06.379795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:06.380696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:06.381445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:06.381445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:06.381445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:06.381445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:06.381445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:06.381445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:06.381445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:06.381445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:06.381445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:06.381445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:06.381445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:06.381445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:06.383132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 63/138] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:06.383329 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3004c7-8243-4751-a9e4-643ed6296326 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:06.402005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d6777-4813-4772-b3bf-38df7e0225c8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:06.434717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:06.441704 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04fd30-5cfa-4ee9-b0e8-1bc72a3744d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:06.443073 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04fd30-5cfa-4ee9-b0e8-1bc72a3744d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:06.443570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04fd30-5cfa-4ee9-b0e8-1bc72a3744d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:06.444066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04fd30-5cfa-4ee9-b0e8-1bc72a3744d5 None admin] PATH_INFO: `/v3/projects/361fcd58851a4377a4e4d372ef8c8c83/users/fb028e92a3234945a85d5d498019b5ce/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:06.446408 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04fd30-5cfa-4ee9-b0e8-1bc72a3744d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=361fcd58851a4377a4e4d372ef8c8c83, user_id=fb028e92a3234945a85d5d498019b5ce, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:06.448164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04fd30-5cfa-4ee9-b0e8-1bc72a3744d5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:06.471651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 76/139] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:55:06 2022] PUT /identity/v3/projects/361fcd58851a4377a4e4d372ef8c8c83/users/fb028e92a3234945a85d5d498019b5ce/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:55:06.473190 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04fd30-5cfa-4ee9-b0e8-1bc72a3744d5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:06.588470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3004c7-8243-4751-a9e4-643ed6296326 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:06.660315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e979342-c9ee-4681-a0fa-0158eec7dda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:06.661787 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e979342-c9ee-4681-a0fa-0158eec7dda3 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:06.662284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e979342-c9ee-4681-a0fa-0158eec7dda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:06.662844 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e979342-c9ee-4681-a0fa-0158eec7dda3 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd/users/5875b90a72824e1280ee921aaccb7829/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:06.665532 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e979342-c9ee-4681-a0fa-0158eec7dda3 None admin] RBAC: Authorizing `identity:create_grant(project_id=27186cff1e94461099e5d6cc062b1abd, user_id=5875b90a72824e1280ee921aaccb7829, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:06.667945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e979342-c9ee-4681-a0fa-0158eec7dda3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:06.684877 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 64/140] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:55:06 2022] PUT /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd/users/5875b90a72824e1280ee921aaccb7829/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:55:06.707427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04fd30-5cfa-4ee9-b0e8-1bc72a3744d5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:06.749315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22f232-49ab-4370-b1c9-627aae006d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:06.750468 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22f232-49ab-4370-b1c9-627aae006d4d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:06.750956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22f232-49ab-4370-b1c9-627aae006d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:06.751526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22f232-49ab-4370-b1c9-627aae006d4d None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:06.752753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22f232-49ab-4370-b1c9-627aae006d4d None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:06.753277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22f232-49ab-4370-b1c9-627aae006d4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:06.755185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22f232-49ab-4370-b1c9-627aae006d4d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:06.766533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 77/141] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:06.772991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e979342-c9ee-4681-a0fa-0158eec7dda3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:06.808013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:06.912727 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e979342-c9ee-4681-a0fa-0158eec7dda3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:06.957408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb249ef-f7ce-449f-b8e8-23fd09e691f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:06.958348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb249ef-f7ce-449f-b8e8-23fd09e691f4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:06.958634 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb249ef-f7ce-449f-b8e8-23fd09e691f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:06.958913 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb249ef-f7ce-449f-b8e8-23fd09e691f4 None admin] PATH_INFO: `/v3/users/fb028e92a3234945a85d5d498019b5ce` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:06.960158 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb249ef-f7ce-449f-b8e8-23fd09e691f4 None admin] RBAC: Authorizing `identity:get_user(user_id=fb028e92a3234945a85d5d498019b5ce)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:06.961668 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb249ef-f7ce-449f-b8e8-23fd09e691f4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:06.963352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 65/142] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:06 2022] GET /identity/v3/users/fb028e92a3234945a85d5d498019b5ce => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:06.969182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22f232-49ab-4370-b1c9-627aae006d4d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:07.062796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:07.130783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22f232-49ab-4370-b1c9-627aae006d4d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:07.167787 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd29131-5b1d-4fb8-bc9e-feebe9a3b367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:07.168369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd29131-5b1d-4fb8-bc9e-feebe9a3b367 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:07.168654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd29131-5b1d-4fb8-bc9e-feebe9a3b367 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:07.168925 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd29131-5b1d-4fb8-bc9e-feebe9a3b367 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:07.256537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd29131-5b1d-4fb8-bc9e-feebe9a3b367 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:07.258047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd29131-5b1d-4fb8-bc9e-feebe9a3b367 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:07.324931 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055351d-dd0e-4ae9-88a2-f7f4c0f6b74b None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:07.325314 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055351d-dd0e-4ae9-88a2-f7f4c0f6b74b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:07.325496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055351d-dd0e-4ae9-88a2-f7f4c0f6b74b None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:07.327054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 66/143] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:07.331974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da19d2-4247-4002-ac18-8a3ce9892368 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:07.332262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da19d2-4247-4002-ac18-8a3ce9892368 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:07.332542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da19d2-4247-4002-ac18-8a3ce9892368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:07.338073 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:07.374733 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e0da19d2-4247-4002-ac18-8a3ce9892368 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:07.375955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e0da19d2-4247-4002-ac18-8a3ce9892368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:07.393749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 78/144] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:07.399555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd29131-5b1d-4fb8-bc9e-feebe9a3b367 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:07.552410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 67/145] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:07.606760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da19d2-4247-4002-ac18-8a3ce9892368 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:07.627025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd29131-5b1d-4fb8-bc9e-feebe9a3b367 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:07.638362 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:07.739202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941f87d-3b03-4c54-99d9-ebeae5d58cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:07.740482 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941f87d-3b03-4c54-99d9-ebeae5d58cdb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:07.740960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941f87d-3b03-4c54-99d9-ebeae5d58cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:07.741426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941f87d-3b03-4c54-99d9-ebeae5d58cdb None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:07.746414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941f87d-3b03-4c54-99d9-ebeae5d58cdb None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:07.748384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941f87d-3b03-4c54-99d9-ebeae5d58cdb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:07.750376 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 79/146] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:07 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:07.759995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941f87d-3b03-4c54-99d9-ebeae5d58cdb None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:07.813767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:07.903353 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da19d2-4247-4002-ac18-8a3ce9892368 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:07.960056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941f87d-3b03-4c54-99d9-ebeae5d58cdb None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:07.995704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead14097-3a97-4857-97db-3e1498169688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:07.996942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14097-3a97-4857-97db-3e1498169688 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:07.997431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14097-3a97-4857-97db-3e1498169688 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:07.997926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14097-3a97-4857-97db-3e1498169688 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:08.006612 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead14097-3a97-4857-97db-3e1498169688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:08.008520 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead14097-3a97-4857-97db-3e1498169688 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:08.010633 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 68/147] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:08.012260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0261d-c09c-48e2-ad6c-82b607cc75a8 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:08.012783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0261d-c09c-48e2-ad6c-82b607cc75a8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:08.013265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0261d-c09c-48e2-ad6c-82b607cc75a8 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:08.015246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 69/148] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:08.021252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0261d-c09c-48e2-ad6c-82b607cc75a8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:08.106533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95013f-dff3-4bb4-b53d-547542e4f6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:08.107944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95013f-dff3-4bb4-b53d-547542e4f6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:08.108470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95013f-dff3-4bb4-b53d-547542e4f6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:08.108950 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95013f-dff3-4bb4-b53d-547542e4f6c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:08.110090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95013f-dff3-4bb4-b53d-547542e4f6c8 None admin] RBAC: Adding query filter params (role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d, scope.project.id=27186cff1e94461099e5d6cc062b1abd, user.id=fb028e92a3234945a85d5d498019b5ce) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:08.110654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95013f-dff3-4bb4-b53d-547542e4f6c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:08.112848 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95013f-dff3-4bb4-b53d-547542e4f6c8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:08.130242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 80/149] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:07 2022] GET /identity/v3/role_assignments?user.id=fb028e92a3234945a85d5d498019b5ce&scope.project.id=27186cff1e94461099e5d6cc062b1abd&role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 608 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:08.132846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019498f-cfba-420e-9cd1-97f76b761d13 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:08.133384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019498f-cfba-420e-9cd1-97f76b761d13 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:08.134028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019498f-cfba-420e-9cd1-97f76b761d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:08.174928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9019498f-cfba-420e-9cd1-97f76b761d13 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:08.176166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9019498f-cfba-420e-9cd1-97f76b761d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:08.202039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:08.258909 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0261d-c09c-48e2-ad6c-82b607cc75a8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:08.340664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 81/150] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:08.342376 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2577e5-55a5-47a1-9d14-cbb95cc399a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:08.343168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2577e5-55a5-47a1-9d14-cbb95cc399a3 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:08.343549 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2577e5-55a5-47a1-9d14-cbb95cc399a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:08.343984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2577e5-55a5-47a1-9d14-cbb95cc399a3 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:08.344813 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2577e5-55a5-47a1-9d14-cbb95cc399a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:08.346676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2577e5-55a5-47a1-9d14-cbb95cc399a3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:08.373671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 70/151] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:55:08 2022] POST /identity/v3/users => generated 283 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:55:08.380764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019498f-cfba-420e-9cd1-97f76b761d13 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:08.402023 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:08.514916 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019498f-cfba-420e-9cd1-97f76b761d13 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:08.551778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15b912-155b-49e2-8f07-0633cb353a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:08.552595 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15b912-155b-49e2-8f07-0633cb353a68 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:08.552873 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15b912-155b-49e2-8f07-0633cb353a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:08.553147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15b912-155b-49e2-8f07-0633cb353a68 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:08.564373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15b912-155b-49e2-8f07-0633cb353a68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-bb15b912-155b-49e2-8f07-0633cb353a68 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:08.572436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:08.573403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:08.574930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:08.574930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:08.574930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:08.574930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:08.574930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:08.574930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:08.574930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:08.574930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:08.574930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:08.574930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:08.574930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:08.574930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:08.574930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 82/152] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:08.580215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2577e5-55a5-47a1-9d14-cbb95cc399a3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:08.692200 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2577e5-55a5-47a1-9d14-cbb95cc399a3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:08.728330 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e031f2-0140-4b15-aefc-2181dce538f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:08.729023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e031f2-0140-4b15-aefc-2181dce538f4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:08.729308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e031f2-0140-4b15-aefc-2181dce538f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:08.729588 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e031f2-0140-4b15-aefc-2181dce538f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:08.730395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e031f2-0140-4b15-aefc-2181dce538f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:08.730701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e031f2-0140-4b15-aefc-2181dce538f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:08.732124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e031f2-0140-4b15-aefc-2181dce538f4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:08.735152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:08.743191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 71/153] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:08.757647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15b912-155b-49e2-8f07-0633cb353a68 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:08.850906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:08.875864 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15b912-155b-49e2-8f07-0633cb353a68 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:08.912120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192e711-d70b-4265-b0ac-0b10d70a879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:08.913019 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192e711-d70b-4265-b0ac-0b10d70a879a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:08.913301 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192e711-d70b-4265-b0ac-0b10d70a879a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:08.913577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192e711-d70b-4265-b0ac-0b10d70a879a None admin] PATH_INFO: `/v3/users/dde6b24774644a72ae522439f0e1e857` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:08.914835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192e711-d70b-4265-b0ac-0b10d70a879a None admin] RBAC: Authorizing `identity:get_user(user_id=dde6b24774644a72ae522439f0e1e857)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:08.916317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192e711-d70b-4265-b0ac-0b10d70a879a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:08.918007 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 83/154] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:08 2022] GET /identity/v3/users/dde6b24774644a72ae522439f0e1e857 => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:08.924153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e031f2-0140-4b15-aefc-2181dce538f4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:09.027429 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e031f2-0140-4b15-aefc-2181dce538f4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:09.061821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dd1a3-35b7-4553-8b00-29b468afa6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:09.062752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dd1a3-35b7-4553-8b00-29b468afa6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:09.063038 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dd1a3-35b7-4553-8b00-29b468afa6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:09.063315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dd1a3-35b7-4553-8b00-29b468afa6f2 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:09.064674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648dd1a3-35b7-4553-8b00-29b468afa6f2 None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:09.066245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648dd1a3-35b7-4553-8b00-29b468afa6f2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:09.067971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 72/155] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:08 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:09.073827 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192e711-d70b-4265-b0ac-0b10d70a879a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:09.149575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3676c-f3eb-4f55-bc63-7a63a9d52c61 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:09.149940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3676c-f3eb-4f55-bc63-7a63a9d52c61 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:09.150199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3676c-f3eb-4f55-bc63-7a63a9d52c61 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:09.151580 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 73/156] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:09.157015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb55ebf-7338-4afb-ba68-fe7ac7f28c92 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:09.157319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb55ebf-7338-4afb-ba68-fe7ac7f28c92 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:09.157785 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb55ebf-7338-4afb-ba68-fe7ac7f28c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:09.177142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2bb55ebf-7338-4afb-ba68-fe7ac7f28c92 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:09.178355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2bb55ebf-7338-4afb-ba68-fe7ac7f28c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:09.190557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192e711-d70b-4265-b0ac-0b10d70a879a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:09.237652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bb8e8-3d9d-4724-82a5-44452839d04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:09.239034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bb8e8-3d9d-4724-82a5-44452839d04b None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:09.239317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bb8e8-3d9d-4724-82a5-44452839d04b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:09.239640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bb8e8-3d9d-4724-82a5-44452839d04b None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd/users/dde6b24774644a72ae522439f0e1e857/roles/f420d4b9bfb64b0f8cd462e9d72bd24d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:09.242223 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bb8e8-3d9d-4724-82a5-44452839d04b None admin] RBAC: Authorizing `identity:create_grant(project_id=27186cff1e94461099e5d6cc062b1abd, user_id=dde6b24774644a72ae522439f0e1e857, role_id=f420d4b9bfb64b0f8cd462e9d72bd24d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:09.244969 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bb8e8-3d9d-4724-82a5-44452839d04b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:09.254237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:09.263657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 84/157] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:55:09 2022] PUT /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd/users/dde6b24774644a72ae522439f0e1e857/roles/f420d4b9bfb64b0f8cd462e9d72bd24d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:55:09.265205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831af36f-ae53-4929-b5fe-5790569d60b1 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:09.265715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831af36f-ae53-4929-b5fe-5790569d60b1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:09.265823 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 74/158] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:09.265917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831af36f-ae53-4929-b5fe-5790569d60b1 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:09.271576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 85/159] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:09.277850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aac158-c0b2-4dce-bf68-bace0847f99b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:09.278158 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aac158-c0b2-4dce-bf68-bace0847f99b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:09.278392 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aac158-c0b2-4dce-bf68-bace0847f99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:09.298770 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-96aac158-c0b2-4dce-bf68-bace0847f99b None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:09.300199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-96aac158-c0b2-4dce-bf68-bace0847f99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:09.300591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831af36f-ae53-4929-b5fe-5790569d60b1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:09.304807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:09.401296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 75/160] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:09.425910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aac158-c0b2-4dce-bf68-bace0847f99b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:09.452245 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831af36f-ae53-4929-b5fe-5790569d60b1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:09.490630 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383226e-9d29-4447-b973-a3b035d868e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:09.491553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383226e-9d29-4447-b973-a3b035d868e2 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:09.491727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383226e-9d29-4447-b973-a3b035d868e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:09.492013 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383226e-9d29-4447-b973-a3b035d868e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:09.506919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383226e-9d29-4447-b973-a3b035d868e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-7383226e-9d29-4447-b973-a3b035d868e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:09.515574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:09.516499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:09.518645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:09.518645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:09.518645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:09.518645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:09.518645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:09.518645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:09.518645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:09.518645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:09.518645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:09.518645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:09.518645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:09.518645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:09.518645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 86/161] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:55:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:09.523259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383226e-9d29-4447-b973-a3b035d868e2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:09.562405 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aac158-c0b2-4dce-bf68-bace0847f99b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:09.592512 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:09.599075 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:09.612933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bb6cd-6208-454b-8e86-396a90d2f14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:09.613829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb6cd-6208-454b-8e86-396a90d2f14e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:09.614158 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb6cd-6208-454b-8e86-396a90d2f14e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:09.615075 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb6cd-6208-454b-8e86-396a90d2f14e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:09.631307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4bb6cd-6208-454b-8e86-396a90d2f14e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-ee4bb6cd-6208-454b-8e86-396a90d2f14e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:09.641635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:09.642574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:09.643548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:09.643548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:09.643548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:09.643548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:09.643548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:09.643548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:09.643548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:09.643548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:09.643548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:09.643548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:09.643548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:09.643548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:09.644829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 76/162] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:09.648814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383226e-9d29-4447-b973-a3b035d868e2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:09.649947 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bb6cd-6208-454b-8e86-396a90d2f14e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:09.694947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838aabe9-feb5-4bfa-906f-d945ca236d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:09.696333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838aabe9-feb5-4bfa-906f-d945ca236d0a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:09.696612 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838aabe9-feb5-4bfa-906f-d945ca236d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:09.696881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838aabe9-feb5-4bfa-906f-d945ca236d0a None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:09.697873 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838aabe9-feb5-4bfa-906f-d945ca236d0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:09.698180 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838aabe9-feb5-4bfa-906f-d945ca236d0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:09.700868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838aabe9-feb5-4bfa-906f-d945ca236d0a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:09.716463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 87/163] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:09.724346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838aabe9-feb5-4bfa-906f-d945ca236d0a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:09.784385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bb6cd-6208-454b-8e86-396a90d2f14e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:09.825516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be728847-04e0-458c-a10c-a7cbba854f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:09.826172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be728847-04e0-458c-a10c-a7cbba854f93 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:09.826451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be728847-04e0-458c-a10c-a7cbba854f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:09.826758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be728847-04e0-458c-a10c-a7cbba854f93 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:09.827879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be728847-04e0-458c-a10c-a7cbba854f93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:09.827879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be728847-04e0-458c-a10c-a7cbba854f93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:09.829322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be728847-04e0-458c-a10c-a7cbba854f93 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:09.839768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 77/164] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:09.845325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be728847-04e0-458c-a10c-a7cbba854f93 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:09.859617 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838aabe9-feb5-4bfa-906f-d945ca236d0a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:09.892099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:09.906418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87051ea6-b5b5-4933-ab86-b60f0224dfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:09.907256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:09.908041 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87051ea6-b5b5-4933-ab86-b60f0224dfad None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:09.908775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87051ea6-b5b5-4933-ab86-b60f0224dfad None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:09.909153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87051ea6-b5b5-4933-ab86-b60f0224dfad None admin] PATH_INFO: `/v3/users/fb028e92a3234945a85d5d498019b5ce` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:09.910835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87051ea6-b5b5-4933-ab86-b60f0224dfad None admin] RBAC: Authorizing `identity:get_user(user_id=fb028e92a3234945a85d5d498019b5ce)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:09.912338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87051ea6-b5b5-4933-ab86-b60f0224dfad None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:09.914676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 88/165] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:09 2022] GET /identity/v3/users/fb028e92a3234945a85d5d498019b5ce => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:09.918585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf87d21-f95f-4e8a-893b-0aa879fadef7 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:09.919732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf87d21-f95f-4e8a-893b-0aa879fadef7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:09.920291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf87d21-f95f-4e8a-893b-0aa879fadef7 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:09.923096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 89/166] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:09.925035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf87d21-f95f-4e8a-893b-0aa879fadef7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:09.984851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be728847-04e0-458c-a10c-a7cbba854f93 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:10.038579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c44526-aaa7-4b91-be5c-466501a7dab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:10.039790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c44526-aaa7-4b91-be5c-466501a7dab8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:10.040270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c44526-aaa7-4b91-be5c-466501a7dab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:10.040732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c44526-aaa7-4b91-be5c-466501a7dab8 None admin] PATH_INFO: `/v3/users/fb028e92a3234945a85d5d498019b5ce` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:10.046428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c44526-aaa7-4b91-be5c-466501a7dab8 None admin] RBAC: Authorizing `identity:get_user(user_id=fb028e92a3234945a85d5d498019b5ce)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:10.047932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c44526-aaa7-4b91-be5c-466501a7dab8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:10.049700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 78/167] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:09 2022] GET /identity/v3/users/fb028e92a3234945a85d5d498019b5ce => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:10.050907 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb87c51-23a4-4919-9d34-cd1f02dccd67 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:10.051191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb87c51-23a4-4919-9d34-cd1f02dccd67 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:10.058423 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb87c51-23a4-4919-9d34-cd1f02dccd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:10.065899 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf87d21-f95f-4e8a-893b-0aa879fadef7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:10.098774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-eeb87c51-23a4-4919-9d34-cd1f02dccd67 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:10.099895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-eeb87c51-23a4-4919-9d34-cd1f02dccd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:10.135134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e45402-0548-465c-a273-c9b1c3be9029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:10.135829 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e45402-0548-465c-a273-c9b1c3be9029 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:10.136119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e45402-0548-465c-a273-c9b1c3be9029 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:10.142350 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e45402-0548-465c-a273-c9b1c3be9029 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:10.230120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:10.249930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e45402-0548-465c-a273-c9b1c3be9029 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:10.252613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e45402-0548-465c-a273-c9b1c3be9029 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:10.264121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 79/168] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:10.264872 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb87c51-23a4-4919-9d34-cd1f02dccd67 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:10.342007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:10.444333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb87c51-23a4-4919-9d34-cd1f02dccd67 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:10.483244 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 90/169] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:10.484112 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e45402-0548-465c-a273-c9b1c3be9029 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:10.523359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874cadd-7c1d-4e14-a472-93959311d117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:10.524180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874cadd-7c1d-4e14-a472-93959311d117 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:10.524468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874cadd-7c1d-4e14-a472-93959311d117 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:10.524746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874cadd-7c1d-4e14-a472-93959311d117 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:10.638906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874cadd-7c1d-4e14-a472-93959311d117 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:10.640681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874cadd-7c1d-4e14-a472-93959311d117 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:10.730101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:10.730976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:10.759777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e45402-0548-465c-a273-c9b1c3be9029 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:10.838648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 80/170] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:10.839873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874cadd-7c1d-4e14-a472-93959311d117 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:10.870312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2c047-8cf4-4187-89a1-6dae1e6e5e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:10.871119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2c047-8cf4-4187-89a1-6dae1e6e5e91 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:10.871396 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2c047-8cf4-4187-89a1-6dae1e6e5e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:10.871673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2c047-8cf4-4187-89a1-6dae1e6e5e91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:10.902522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2c047-8cf4-4187-89a1-6dae1e6e5e91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-95f2c047-8cf4-4187-89a1-6dae1e6e5e91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:10.910453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:10.911353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:10.912133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:10.912133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:10.912133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:10.912133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:10.912133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:10.912133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:10.912133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:10.912133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:10.912133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:10.912133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:10.912133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:10.912133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:10.912133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 91/171] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:55:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:10.912870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2c047-8cf4-4187-89a1-6dae1e6e5e91 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:11.047969 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874cadd-7c1d-4e14-a472-93959311d117 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:11.109508 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493f514-a0db-46ec-8729-14b99b5a0f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:11.110503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493f514-a0db-46ec-8729-14b99b5a0f1a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:11.110847 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493f514-a0db-46ec-8729-14b99b5a0f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:11.111140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493f514-a0db-46ec-8729-14b99b5a0f1a None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:11.114593 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493f514-a0db-46ec-8729-14b99b5a0f1a None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:11.116230 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493f514-a0db-46ec-8729-14b99b5a0f1a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:11.117993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 81/172] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:10 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:11.119160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493f514-a0db-46ec-8729-14b99b5a0f1a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:11.158164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:11.182984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2c047-8cf4-4187-89a1-6dae1e6e5e91 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:11.262201 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87adfdd-0f53-4c5a-a16a-9af7bd39218e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:11.263110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87adfdd-0f53-4c5a-a16a-9af7bd39218e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:11.263390 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87adfdd-0f53-4c5a-a16a-9af7bd39218e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:11.263663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87adfdd-0f53-4c5a-a16a-9af7bd39218e None admin] PATH_INFO: `/v3/projects/361fcd58851a4377a4e4d372ef8c8c83` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:11.264979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87adfdd-0f53-4c5a-a16a-9af7bd39218e None admin] RBAC: Authorizing `identity:get_project(project_id=361fcd58851a4377a4e4d372ef8c8c83)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:11.266483 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87adfdd-0f53-4c5a-a16a-9af7bd39218e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:11.268475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 92/173] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:10 2022] GET /identity/v3/projects/361fcd58851a4377a4e4d372ef8c8c83 => generated 312 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:11.269267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87adfdd-0f53-4c5a-a16a-9af7bd39218e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:11.322853 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:11.356199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493f514-a0db-46ec-8729-14b99b5a0f1a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:11.415453 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0adf0-5dac-422e-89df-f455cc2cdcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:11.416165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0adf0-5dac-422e-89df-f455cc2cdcdb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:11.416455 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0adf0-5dac-422e-89df-f455cc2cdcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:11.416736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0adf0-5dac-422e-89df-f455cc2cdcdb None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:11.417512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0adf0-5dac-422e-89df-f455cc2cdcdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:11.417861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0adf0-5dac-422e-89df-f455cc2cdcdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:11.419305 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0adf0-5dac-422e-89df-f455cc2cdcdb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:11.437760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 82/174] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:11.440264 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0adf0-5dac-422e-89df-f455cc2cdcdb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:11.458591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87adfdd-0f53-4c5a-a16a-9af7bd39218e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:11.526387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1a077-cee1-4786-826e-1daca1e8b07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:11.527135 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1a077-cee1-4786-826e-1daca1e8b07d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:11.527423 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1a077-cee1-4786-826e-1daca1e8b07d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:11.527705 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1a077-cee1-4786-826e-1daca1e8b07d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:11.528618 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1a077-cee1-4786-826e-1daca1e8b07d None admin] RBAC: Adding query filter params (role.id=78a15fc7951d4f28beb57f54dace457d, scope.project.id=27186cff1e94461099e5d6cc062b1abd, user.id=fb028e92a3234945a85d5d498019b5ce) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:11.528979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1a077-cee1-4786-826e-1daca1e8b07d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:11.530475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1a077-cee1-4786-826e-1daca1e8b07d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:11.543388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 93/175] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:11 2022] GET /identity/v3/role_assignments?user.id=fb028e92a3234945a85d5d498019b5ce&scope.project.id=27186cff1e94461099e5d6cc062b1abd&role.id=78a15fc7951d4f28beb57f54dace457d => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:11.544223 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1a077-cee1-4786-826e-1daca1e8b07d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:11.578131 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:11.607867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0adf0-5dac-422e-89df-f455cc2cdcdb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:11.665167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b3907-2527-4d0f-89b3-b9f5f3670dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:11.665862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b3907-2527-4d0f-89b3-b9f5f3670dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:11.666145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b3907-2527-4d0f-89b3-b9f5f3670dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:11.666422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b3907-2527-4d0f-89b3-b9f5f3670dd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:11.667269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b3907-2527-4d0f-89b3-b9f5f3670dd8 None admin] RBAC: Adding query filter params (role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d, scope.project.id=361fcd58851a4377a4e4d372ef8c8c83, user.id=fb028e92a3234945a85d5d498019b5ce) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:11.667577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b3907-2527-4d0f-89b3-b9f5f3670dd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:11.670262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b3907-2527-4d0f-89b3-b9f5f3670dd8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:11.683860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 83/176] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:11 2022] GET /identity/v3/role_assignments?user.id=fb028e92a3234945a85d5d498019b5ce&scope.project.id=361fcd58851a4377a4e4d372ef8c8c83&role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:11.732671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:11.750274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1a077-cee1-4786-826e-1daca1e8b07d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:11.785854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5c219-3045-4e8a-8a59-dbe2b695f5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:11.786820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5c219-3045-4e8a-8a59-dbe2b695f5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:11.787105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5c219-3045-4e8a-8a59-dbe2b695f5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:11.787431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5c219-3045-4e8a-8a59-dbe2b695f5a0 None admin] PATH_INFO: `/v3/users/5875b90a72824e1280ee921aaccb7829` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:11.788748 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5c219-3045-4e8a-8a59-dbe2b695f5a0 None admin] RBAC: Authorizing `identity:get_user(user_id=5875b90a72824e1280ee921aaccb7829)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:11.790307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5c219-3045-4e8a-8a59-dbe2b695f5a0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:11.792068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 94/177] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:11 2022] GET /identity/v3/users/5875b90a72824e1280ee921aaccb7829 => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:11.798036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b3907-2527-4d0f-89b3-b9f5f3670dd8 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:11.931945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b3907-2527-4d0f-89b3-b9f5f3670dd8 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:11.975154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe15421-9bd0-426f-92aa-e7e0102f5035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:11.976171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe15421-9bd0-426f-92aa-e7e0102f5035 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:11.976459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe15421-9bd0-426f-92aa-e7e0102f5035 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:11.976735 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe15421-9bd0-426f-92aa-e7e0102f5035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:11.990085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:12.055487 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe15421-9bd0-426f-92aa-e7e0102f5035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:12.057002 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe15421-9bd0-426f-92aa-e7e0102f5035 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:12.098526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04944c-1aad-4dfa-90e5-c4ae1b4e0e5d None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:12.098836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04944c-1aad-4dfa-90e5-c4ae1b4e0e5d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:12.099073 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04944c-1aad-4dfa-90e5-c4ae1b4e0e5d None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:12.100447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 95/178] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:12.106349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476dccf-1fa8-492d-aaf8-d6da27313390 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:12.106644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476dccf-1fa8-492d-aaf8-d6da27313390 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:12.106936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476dccf-1fa8-492d-aaf8-d6da27313390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:12.127553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b476dccf-1fa8-492d-aaf8-d6da27313390 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:12.129235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b476dccf-1fa8-492d-aaf8-d6da27313390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:12.193278 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 84/179] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:12.202978 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe15421-9bd0-426f-92aa-e7e0102f5035 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:12.244913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 96/180] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:12.255276 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:12.336211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe15421-9bd0-426f-92aa-e7e0102f5035 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:12.346037 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476dccf-1fa8-492d-aaf8-d6da27313390 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:12.381688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37f898-bd1b-4bba-9d66-56927c1cdbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:12.381688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37f898-bd1b-4bba-9d66-56927c1cdbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:12.381688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37f898-bd1b-4bba-9d66-56927c1cdbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:12.381688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37f898-bd1b-4bba-9d66-56927c1cdbc9 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:12.382881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37f898-bd1b-4bba-9d66-56927c1cdbc9 None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:12.384546 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37f898-bd1b-4bba-9d66-56927c1cdbc9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:12.386784 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 85/181] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:12 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:12.397520 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37f898-bd1b-4bba-9d66-56927c1cdbc9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:12.473343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:12.509213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476dccf-1fa8-492d-aaf8-d6da27313390 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:12.522893 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37f898-bd1b-4bba-9d66-56927c1cdbc9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:12.533675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:12.559786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63177fa7-ab69-4f98-b094-eac0a47ef0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:12.560704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63177fa7-ab69-4f98-b094-eac0a47ef0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:12.561419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63177fa7-ab69-4f98-b094-eac0a47ef0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:12.561890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63177fa7-ab69-4f98-b094-eac0a47ef0b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:12.562990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63177fa7-ab69-4f98-b094-eac0a47ef0b3 None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=27186cff1e94461099e5d6cc062b1abd, user.id=5875b90a72824e1280ee921aaccb7829) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:12.563444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63177fa7-ab69-4f98-b094-eac0a47ef0b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:12.565589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63177fa7-ab69-4f98-b094-eac0a47ef0b3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:12.571906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 86/182] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:12 2022] GET /identity/v3/role_assignments?user.id=5875b90a72824e1280ee921aaccb7829&scope.project.id=27186cff1e94461099e5d6cc062b1abd&role.id=a3de9df253294d069dc1867fa26600c3 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:12.589111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e5c96-8f5f-4fc0-b746-70c0f07dc707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:12.589978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e5c96-8f5f-4fc0-b746-70c0f07dc707 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:12.590265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e5c96-8f5f-4fc0-b746-70c0f07dc707 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:12.590545 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e5c96-8f5f-4fc0-b746-70c0f07dc707 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:12.613186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e5c96-8f5f-4fc0-b746-70c0f07dc707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d90e5c96-8f5f-4fc0-b746-70c0f07dc707 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:12.620953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:12.622053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:12.622980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:12.622980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:12.622980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:12.622980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:12.622980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:12.622980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:12.622980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:12.622980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:12.622980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:12.622980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:12.622980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:12.622980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:12.622980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 97/183] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:55:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:12.628324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63177fa7-ab69-4f98-b094-eac0a47ef0b3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:12.684568 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a837c02-4677-4240-853a-76dd5a733181 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:12.685126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a837c02-4677-4240-853a-76dd5a733181 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:12.685551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a837c02-4677-4240-853a-76dd5a733181 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:12.689739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 98/184] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:12.695159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce512d8-b4af-46a2-b634-5c01a10365d9 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:12.695673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce512d8-b4af-46a2-b634-5c01a10365d9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:12.695956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce512d8-b4af-46a2-b634-5c01a10365d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:12.738936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-bce512d8-b4af-46a2-b634-5c01a10365d9 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:12.740079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bce512d8-b4af-46a2-b634-5c01a10365d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:12.778520 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63177fa7-ab69-4f98-b094-eac0a47ef0b3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:12.831553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320fda8-f1e7-400a-8ecd-a29acc5b2b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:12.832253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320fda8-f1e7-400a-8ecd-a29acc5b2b0e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:12.832532 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320fda8-f1e7-400a-8ecd-a29acc5b2b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:12.832808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320fda8-f1e7-400a-8ecd-a29acc5b2b0e None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:12.833699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320fda8-f1e7-400a-8ecd-a29acc5b2b0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:12.834008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320fda8-f1e7-400a-8ecd-a29acc5b2b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:12.835521 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320fda8-f1e7-400a-8ecd-a29acc5b2b0e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:12.845506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 87/185] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:12.854921 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320fda8-f1e7-400a-8ecd-a29acc5b2b0e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:12.864855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 99/186] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:12.866672 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f271b-9b23-4d5f-aeb3-5779e88b2b80 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:12.867106 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f271b-9b23-4d5f-aeb3-5779e88b2b80 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:12.867399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f271b-9b23-4d5f-aeb3-5779e88b2b80 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:12.868813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 100/187] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:12.876291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43e7bd-f3a1-4385-a183-5a2eb1c5025a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:12.876585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43e7bd-f3a1-4385-a183-5a2eb1c5025a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:12.876858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43e7bd-f3a1-4385-a183-5a2eb1c5025a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:12.896221 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:12.897575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3b43e7bd-f3a1-4385-a183-5a2eb1c5025a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:12.899711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3b43e7bd-f3a1-4385-a183-5a2eb1c5025a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:12.971760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320fda8-f1e7-400a-8ecd-a29acc5b2b0e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:12.974578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 101/188] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:12.975431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43e7bd-f3a1-4385-a183-5a2eb1c5025a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:12.996063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:13.006839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db8c7b-6ba5-4dd2-b84d-11fce7386a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:13.007737 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db8c7b-6ba5-4dd2-b84d-11fce7386a8b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:13.008020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db8c7b-6ba5-4dd2-b84d-11fce7386a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:13.008296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db8c7b-6ba5-4dd2-b84d-11fce7386a8b None admin] PATH_INFO: `/v3/users/442acb89c935401da075802a768b8ca3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:13.022211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db8c7b-6ba5-4dd2-b84d-11fce7386a8b None admin] RBAC: Authorizing `identity:get_user(user_id=442acb89c935401da075802a768b8ca3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:13.023714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db8c7b-6ba5-4dd2-b84d-11fce7386a8b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:13.025567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 88/189] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:12 2022] GET /identity/v3/users/442acb89c935401da075802a768b8ca3 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:13.026653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c96773-440e-46cd-8bee-c1a2828d8aa9 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:13.026942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c96773-440e-46cd-8bee-c1a2828d8aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:13.027217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c96773-440e-46cd-8bee-c1a2828d8aa9 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:13.028435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 89/190] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:13.029121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c96773-440e-46cd-8bee-c1a2828d8aa9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:13.077528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43e7bd-f3a1-4385-a183-5a2eb1c5025a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:13.111577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b549e-2e68-4b3e-8362-ce4cdcc2e0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:13.112392 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b549e-2e68-4b3e-8362-ce4cdcc2e0ca None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:13.112676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b549e-2e68-4b3e-8362-ce4cdcc2e0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:13.112951 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b549e-2e68-4b3e-8362-ce4cdcc2e0ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:13.121204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:13.123270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b549e-2e68-4b3e-8362-ce4cdcc2e0ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:13.132573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c96773-440e-46cd-8bee-c1a2828d8aa9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c27b549e-2e68-4b3e-8362-ce4cdcc2e0ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:13.132703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:13.133542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:13.134654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:13.134654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:13.134654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:13.134654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:13.134654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:13.134654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:13.134654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:13.134654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:13.134654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:13.134654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:13.134654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:13.134654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:13.134654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 102/191] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:55:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:13.138494 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b549e-2e68-4b3e-8362-ce4cdcc2e0ca None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:13.171151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e68fa9-4300-4748-b389-6ab9796addaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:13.171966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e68fa9-4300-4748-b389-6ab9796addaa None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:13.172249 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e68fa9-4300-4748-b389-6ab9796addaa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:13.172529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e68fa9-4300-4748-b389-6ab9796addaa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:13.183272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e68fa9-4300-4748-b389-6ab9796addaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-71e68fa9-4300-4748-b389-6ab9796addaa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:13.191020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:13.192029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:13.192818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:13.192818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:13.192818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:13.192818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:13.192818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:13.192818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:13.192818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:13.192818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:13.192818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:13.192818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:13.192818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:13.192818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:13.192818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 90/192] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:13.193709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4df289-72fd-4f77-9c27-50aa7101cfe4 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:13.193961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4df289-72fd-4f77-9c27-50aa7101cfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:13.194363 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4df289-72fd-4f77-9c27-50aa7101cfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:13.205786 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:13.210394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1b4df289-72fd-4f77-9c27-50aa7101cfe4 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:13.211587 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1b4df289-72fd-4f77-9c27-50aa7101cfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:13.246050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b549e-2e68-4b3e-8362-ce4cdcc2e0ca None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:13.276764 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 91/193] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:13.277584 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4df289-72fd-4f77-9c27-50aa7101cfe4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:13.279691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9a175-ff7b-40e9-8502-bdb56fd13649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:13.280354 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9a175-ff7b-40e9-8502-bdb56fd13649 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:13.280641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9a175-ff7b-40e9-8502-bdb56fd13649 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:13.280921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9a175-ff7b-40e9-8502-bdb56fd13649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:13.349512 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9a175-ff7b-40e9-8502-bdb56fd13649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:13.351412 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9a175-ff7b-40e9-8502-bdb56fd13649 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:13.372908 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:13.391360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4df289-72fd-4f77-9c27-50aa7101cfe4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:13.410470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:13.424029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74130e-2345-4c85-991f-754f953775d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:13.424744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74130e-2345-4c85-991f-754f953775d8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:13.425017 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74130e-2345-4c85-991f-754f953775d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:13.425290 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74130e-2345-4c85-991f-754f953775d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:13.426069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74130e-2345-4c85-991f-754f953775d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:13.426348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74130e-2345-4c85-991f-754f953775d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:13.427739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74130e-2345-4c85-991f-754f953775d8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:13.435589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 92/194] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:13.436367 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74130e-2345-4c85-991f-754f953775d8 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:13.452823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 103/195] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:13.454019 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9a175-ff7b-40e9-8502-bdb56fd13649 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:13.536490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74130e-2345-4c85-991f-754f953775d8 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:13.552224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9a175-ff7b-40e9-8502-bdb56fd13649 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:13.569007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ac46b-4dbb-480d-8ffb-340eb6c0d25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:13.569705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ac46b-4dbb-480d-8ffb-340eb6c0d25e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:13.569992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ac46b-4dbb-480d-8ffb-340eb6c0d25e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:13.570270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ac46b-4dbb-480d-8ffb-340eb6c0d25e None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:13.571020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ac46b-4dbb-480d-8ffb-340eb6c0d25e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:13.571324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ac46b-4dbb-480d-8ffb-340eb6c0d25e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:13.572725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ac46b-4dbb-480d-8ffb-340eb6c0d25e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:13.581179 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 93/196] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:13.582245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ac46b-4dbb-480d-8ffb-340eb6c0d25e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:13.586131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc48715-6015-454f-a261-02eb5559ec29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:13.586659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc48715-6015-454f-a261-02eb5559ec29 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:13.586944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc48715-6015-454f-a261-02eb5559ec29 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:13.587225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc48715-6015-454f-a261-02eb5559ec29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:13.588440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc48715-6015-454f-a261-02eb5559ec29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:13.589927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc48715-6015-454f-a261-02eb5559ec29 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:13.591062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:13.591543 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 104/197] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:13.592381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc48715-6015-454f-a261-02eb5559ec29 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:13.623623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:13.686411 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ac46b-4dbb-480d-8ffb-340eb6c0d25e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:13.692693 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc48715-6015-454f-a261-02eb5559ec29 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:13.719393 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c4896-ee20-4a67-9054-c015b9e802bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:13.720431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c4896-ee20-4a67-9054-c015b9e802bc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:13.720711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c4896-ee20-4a67-9054-c015b9e802bc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:13.720989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c4896-ee20-4a67-9054-c015b9e802bc None admin] PATH_INFO: `/v3/users/442acb89c935401da075802a768b8ca3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:13.722247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c4896-ee20-4a67-9054-c015b9e802bc None admin] RBAC: Authorizing `identity:get_user(user_id=442acb89c935401da075802a768b8ca3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:13.723719 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c4896-ee20-4a67-9054-c015b9e802bc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:13.725530 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 94/198] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:13 2022] GET /identity/v3/users/442acb89c935401da075802a768b8ca3 => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:13.726579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c4896-ee20-4a67-9054-c015b9e802bc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:13.731396 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bafd3-90b4-41b4-b97b-56bd92e30294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:13.732263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bafd3-90b4-41b4-b97b-56bd92e30294 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:13.732542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bafd3-90b4-41b4-b97b-56bd92e30294 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:13.732815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bafd3-90b4-41b4-b97b-56bd92e30294 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:13.734041 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bafd3-90b4-41b4-b97b-56bd92e30294 None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:13.735582 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bafd3-90b4-41b4-b97b-56bd92e30294 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:13.737305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 105/199] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:13 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:13.738058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bafd3-90b4-41b4-b97b-56bd92e30294 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:13.799691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:13.825753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c4896-ee20-4a67-9054-c015b9e802bc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:13.834183 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:13.838244 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bafd3-90b4-41b4-b97b-56bd92e30294 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:13.859520 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1c6c6-d42f-41b7-a78d-da64a5f3e9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:13.860504 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1c6c6-d42f-41b7-a78d-da64a5f3e9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:13.860849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1c6c6-d42f-41b7-a78d-da64a5f3e9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:13.861123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1c6c6-d42f-41b7-a78d-da64a5f3e9b6 None admin] PATH_INFO: `/v3/users/dde6b24774644a72ae522439f0e1e857` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:13.862616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1c6c6-d42f-41b7-a78d-da64a5f3e9b6 None admin] RBAC: Authorizing `identity:get_user(user_id=dde6b24774644a72ae522439f0e1e857)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:13.864136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1c6c6-d42f-41b7-a78d-da64a5f3e9b6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:13.865831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 95/200] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:13 2022] GET /identity/v3/users/dde6b24774644a72ae522439f0e1e857 => generated 289 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:13.866589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1c6c6-d42f-41b7-a78d-da64a5f3e9b6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:13.873248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713775e-d09d-4bfb-8925-a63875b423dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:13.873962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713775e-d09d-4bfb-8925-a63875b423dc None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:13.874247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713775e-d09d-4bfb-8925-a63875b423dc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:13.874524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713775e-d09d-4bfb-8925-a63875b423dc None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:13.875243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713775e-d09d-4bfb-8925-a63875b423dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:13.877167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713775e-d09d-4bfb-8925-a63875b423dc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:13.896894 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 106/201] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:55:13 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:55:13.898959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713775e-d09d-4bfb-8925-a63875b423dc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:13.980540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1c6c6-d42f-41b7-a78d-da64a5f3e9b6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:14.018709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4827a9-91d7-40b2-8634-1063afaef495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:14.019163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713775e-d09d-4bfb-8925-a63875b423dc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:14.019450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4827a9-91d7-40b2-8634-1063afaef495 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:14.019744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4827a9-91d7-40b2-8634-1063afaef495 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:14.020028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4827a9-91d7-40b2-8634-1063afaef495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:14.034308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:14.056470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82950a58-10ae-4441-853b-a9e49c8fe9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:14.057378 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82950a58-10ae-4441-853b-a9e49c8fe9ee None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:14.057693 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82950a58-10ae-4441-853b-a9e49c8fe9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:14.057974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82950a58-10ae-4441-853b-a9e49c8fe9ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:14.058828 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82950a58-10ae-4441-853b-a9e49c8fe9ee None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=62e3ae8270a64809a9778530c2b3b7f0, user.id=442acb89c935401da075802a768b8ca3) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:14.059139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82950a58-10ae-4441-853b-a9e49c8fe9ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:14.060736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82950a58-10ae-4441-853b-a9e49c8fe9ee None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:14.067721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 107/202] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:13 2022] GET /identity/v3/role_assignments?user.id=442acb89c935401da075802a768b8ca3&scope.project.id=62e3ae8270a64809a9778530c2b3b7f0&role.id=a3de9df253294d069dc1867fa26600c3 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:14.068411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82950a58-10ae-4441-853b-a9e49c8fe9ee None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:14.095462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:14.103957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4827a9-91d7-40b2-8634-1063afaef495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:14.105803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4827a9-91d7-40b2-8634-1063afaef495 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:14.217237 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82950a58-10ae-4441-853b-a9e49c8fe9ee None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:14.252902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 96/203] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:14.258948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4827a9-91d7-40b2-8634-1063afaef495 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:14.269783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8257f-cc20-4ffb-a06a-ed60d31cf414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:14.270452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8257f-cc20-4ffb-a06a-ed60d31cf414 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:14.270740 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8257f-cc20-4ffb-a06a-ed60d31cf414 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:14.271663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8257f-cc20-4ffb-a06a-ed60d31cf414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:14.326129 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:14.380231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4827a9-91d7-40b2-8634-1063afaef495 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:14.401724 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8257f-cc20-4ffb-a06a-ed60d31cf414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:14.404685 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8257f-cc20-4ffb-a06a-ed60d31cf414 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:14.428190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0c81a-5888-4c2c-8bd6-ca177ca1d15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:14.429069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0c81a-5888-4c2c-8bd6-ca177ca1d15e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:14.429351 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0c81a-5888-4c2c-8bd6-ca177ca1d15e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:14.429652 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0c81a-5888-4c2c-8bd6-ca177ca1d15e None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:14.431235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:14.432007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0c81a-5888-4c2c-8bd6-ca177ca1d15e None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:14.433281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0c81a-5888-4c2c-8bd6-ca177ca1d15e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:14.439293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 97/204] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:14 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:14.445709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0c81a-5888-4c2c-8bd6-ca177ca1d15e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:14.581039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 108/205] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:14.588571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8257f-cc20-4ffb-a06a-ed60d31cf414 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:14.617032 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0c81a-5888-4c2c-8bd6-ca177ca1d15e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:14.642467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:14.686779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cecb4-4f7f-4ab7-bebc-9612b4a14999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:14.687451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cecb4-4f7f-4ab7-bebc-9612b4a14999 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:14.687739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cecb4-4f7f-4ab7-bebc-9612b4a14999 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:14.688020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cecb4-4f7f-4ab7-bebc-9612b4a14999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:14.689171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cecb4-4f7f-4ab7-bebc-9612b4a14999 None admin] RBAC: Adding query filter params (role.id=78a15fc7951d4f28beb57f54dace457d, scope.project.id=62e3ae8270a64809a9778530c2b3b7f0, user.id=442acb89c935401da075802a768b8ca3) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:14.689494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cecb4-4f7f-4ab7-bebc-9612b4a14999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:14.691122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cecb4-4f7f-4ab7-bebc-9612b4a14999 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:14.705744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 98/206] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:14 2022] GET /identity/v3/role_assignments?user.id=442acb89c935401da075802a768b8ca3&scope.project.id=62e3ae8270a64809a9778530c2b3b7f0&role.id=78a15fc7951d4f28beb57f54dace457d => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:14.739808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8257f-cc20-4ffb-a06a-ed60d31cf414 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:14.775124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:14.801494 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253b258-0034-4367-abe7-ec275d1b4734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:14.802422 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253b258-0034-4367-abe7-ec275d1b4734 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:14.802710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253b258-0034-4367-abe7-ec275d1b4734 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:14.802988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253b258-0034-4367-abe7-ec275d1b4734 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:14.804785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253b258-0034-4367-abe7-ec275d1b4734 None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:14.806348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253b258-0034-4367-abe7-ec275d1b4734 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:14.808204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 109/207] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:14 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:14.815290 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cecb4-4f7f-4ab7-bebc-9612b4a14999 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:14.945814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cecb4-4f7f-4ab7-bebc-9612b4a14999 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:14.983851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ea618-6b9a-46c0-9b0d-da9d2e5ba4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:14.984532 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ea618-6b9a-46c0-9b0d-da9d2e5ba4bb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:14.984818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ea618-6b9a-46c0-9b0d-da9d2e5ba4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:14.985094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ea618-6b9a-46c0-9b0d-da9d2e5ba4bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:14.986017 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ea618-6b9a-46c0-9b0d-da9d2e5ba4bb None admin] RBAC: Adding query filter params (role.id=f420d4b9bfb64b0f8cd462e9d72bd24d, scope.project.id=27186cff1e94461099e5d6cc062b1abd, user.id=dde6b24774644a72ae522439f0e1e857) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:14.986387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ea618-6b9a-46c0-9b0d-da9d2e5ba4bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:14.987940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ea618-6b9a-46c0-9b0d-da9d2e5ba4bb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:14.994885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 99/208] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:14 2022] GET /identity/v3/role_assignments?user.id=dde6b24774644a72ae522439f0e1e857&scope.project.id=27186cff1e94461099e5d6cc062b1abd&role.id=f420d4b9bfb64b0f8cd462e9d72bd24d => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:16.210805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aded901-2d70-488c-a014-9d73c9bcb73c None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:16.211359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aded901-2d70-488c-a014-9d73c9bcb73c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:16.211359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aded901-2d70-488c-a014-9d73c9bcb73c None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:16.212882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 110/209] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:16.218962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0929bf1-034c-4811-b6fa-90a53ae9a0c3 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:16.218962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0929bf1-034c-4811-b6fa-90a53ae9a0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:16.218962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0929bf1-034c-4811-b6fa-90a53ae9a0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:16.237350 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b0929bf1-034c-4811-b6fa-90a53ae9a0c3 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:16.238923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b0929bf1-034c-4811-b6fa-90a53ae9a0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:16.270595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:16.314347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 100/210] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:16.338399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aded901-2d70-488c-a014-9d73c9bcb73c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:16.399402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac139f47-ccd1-4c34-b970-97d1278e6ad2 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:16.400010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac139f47-ccd1-4c34-b970-97d1278e6ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:16.400486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac139f47-ccd1-4c34-b970-97d1278e6ad2 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:16.402433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 101/211] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:16.407460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff4608-da53-421b-a595-363cfaad4eed None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:16.407777 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff4608-da53-421b-a595-363cfaad4eed None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:16.408040 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff4608-da53-421b-a595-363cfaad4eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:16.425885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-99ff4608-da53-421b-a595-363cfaad4eed None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:16.427406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-99ff4608-da53-421b-a595-363cfaad4eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:16.444560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aded901-2d70-488c-a014-9d73c9bcb73c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:16.480454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c971e8-d92d-424d-9798-262ce9f1238f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:16.481268 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c971e8-d92d-424d-9798-262ce9f1238f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:16.481664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c971e8-d92d-424d-9798-262ce9f1238f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:16.481935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c971e8-d92d-424d-9798-262ce9f1238f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:16.493503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c971e8-d92d-424d-9798-262ce9f1238f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:16.500667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 102/212] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d7c971e8-d92d-424d-9798-262ce9f1238f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:16.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:16.502435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:16.503211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:16.503211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:16.503211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:16.503211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:16.503211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:16.503211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:16.503211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:16.503211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:16.503211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:16.503211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:16.503211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:16.503211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:16.507597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 111/213] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:16.508300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c971e8-d92d-424d-9798-262ce9f1238f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:16.511173 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4511c9-8ca5-4be0-9b6b-b4cdb916ae18 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:16.511470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4511c9-8ca5-4be0-9b6b-b4cdb916ae18 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:16.511755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4511c9-8ca5-4be0-9b6b-b4cdb916ae18 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:16.513271 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 103/214] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:16 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:16.519726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f40778-098a-436a-8e15-0bed626d8632 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:16.520794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f40778-098a-436a-8e15-0bed626d8632 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:16.521280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f40778-098a-436a-8e15-0bed626d8632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:16.539927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-31f40778-098a-436a-8e15-0bed626d8632 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:16.541362 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-31f40778-098a-436a-8e15-0bed626d8632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:16.557353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:16.612226 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 104/215] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:16.613407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f40778-098a-436a-8e15-0bed626d8632 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:16.620662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c971e8-d92d-424d-9798-262ce9f1238f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:16.654247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:16.661728 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea7c47-2ecd-45e0-86eb-defce5d9a6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:16.662424 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea7c47-2ecd-45e0-86eb-defce5d9a6fc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:16.662713 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea7c47-2ecd-45e0-86eb-defce5d9a6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:16.662996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea7c47-2ecd-45e0-86eb-defce5d9a6fc None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:16.663761 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea7c47-2ecd-45e0-86eb-defce5d9a6fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:16.664075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea7c47-2ecd-45e0-86eb-defce5d9a6fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:16.665489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea7c47-2ecd-45e0-86eb-defce5d9a6fc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:16.679483 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 112/216] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:16.680324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea7c47-2ecd-45e0-86eb-defce5d9a6fc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:16.730615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f40778-098a-436a-8e15-0bed626d8632 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:16.765059 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9ecec-e6fd-48dd-b788-fe00f4fe4014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:16.765893 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9ecec-e6fd-48dd-b788-fe00f4fe4014 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:16.766178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9ecec-e6fd-48dd-b788-fe00f4fe4014 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:16.766457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9ecec-e6fd-48dd-b788-fe00f4fe4014 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:16.776976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9ecec-e6fd-48dd-b788-fe00f4fe4014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-33c9ecec-e6fd-48dd-b788-fe00f4fe4014 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:16.784690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:16.785860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 105/217] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:55:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:16.786853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9ecec-e6fd-48dd-b788-fe00f4fe4014 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:16.797928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea7c47-2ecd-45e0-86eb-defce5d9a6fc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:16.808207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:16.837743 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003aab1d-c6e9-47f0-a534-6a4186aa485b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:16.838587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003aab1d-c6e9-47f0-a534-6a4186aa485b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:16.838876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003aab1d-c6e9-47f0-a534-6a4186aa485b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:16.839155 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003aab1d-c6e9-47f0-a534-6a4186aa485b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:16.840342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003aab1d-c6e9-47f0-a534-6a4186aa485b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:16.841840 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003aab1d-c6e9-47f0-a534-6a4186aa485b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:16.848698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 113/218] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:16.848698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003aab1d-c6e9-47f0-a534-6a4186aa485b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:16.884935 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:16.899453 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9ecec-e6fd-48dd-b788-fe00f4fe4014 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:16.935129 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bfb4e-4907-4511-b6ab-4b8a09ddf9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:16.936035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bfb4e-4907-4511-b6ab-4b8a09ddf9bc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:16.936326 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bfb4e-4907-4511-b6ab-4b8a09ddf9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:16.936606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bfb4e-4907-4511-b6ab-4b8a09ddf9bc None admin] PATH_INFO: `/v3/users/53aba92d86724ccf876f3f10db8ada11` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:16.950680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003aab1d-c6e9-47f0-a534-6a4186aa485b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:16.951316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bfb4e-4907-4511-b6ab-4b8a09ddf9bc None admin] RBAC: Authorizing `identity:get_user(user_id=53aba92d86724ccf876f3f10db8ada11)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:16.953706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bfb4e-4907-4511-b6ab-4b8a09ddf9bc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:16.960515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 106/219] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:16 2022] GET /identity/v3/users/53aba92d86724ccf876f3f10db8ada11 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:16.960515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bfb4e-4907-4511-b6ab-4b8a09ddf9bc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:16.985645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c159c-3c6a-4f16-80f4-240743cabad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:16.986563 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c159c-3c6a-4f16-80f4-240743cabad8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:16.986848 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c159c-3c6a-4f16-80f4-240743cabad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:16.987112 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c159c-3c6a-4f16-80f4-240743cabad8 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:16.987865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c159c-3c6a-4f16-80f4-240743cabad8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:16.988173 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c159c-3c6a-4f16-80f4-240743cabad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:16.989562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c159c-3c6a-4f16-80f4-240743cabad8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:16.998142 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 114/220] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:16.998930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c159c-3c6a-4f16-80f4-240743cabad8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:17.037726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:17.065199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bfb4e-4907-4511-b6ab-4b8a09ddf9bc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:17.096476 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c159c-3c6a-4f16-80f4-240743cabad8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:17.099689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286eca4-7ef0-4525-8e3e-87aaafdd784d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:17.100396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286eca4-7ef0-4525-8e3e-87aaafdd784d None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:17.101027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286eca4-7ef0-4525-8e3e-87aaafdd784d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:17.101307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286eca4-7ef0-4525-8e3e-87aaafdd784d None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:17.102051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286eca4-7ef0-4525-8e3e-87aaafdd784d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:17.103497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286eca4-7ef0-4525-8e3e-87aaafdd784d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:17.126962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 107/221] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:55:16 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:55:17.128562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2ca32-7b43-43b7-892a-7563782ab3f3 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:17.128877 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2ca32-7b43-43b7-892a-7563782ab3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:17.129194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2ca32-7b43-43b7-892a-7563782ab3f3 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:17.132696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 108/222] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:17.132696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2ca32-7b43-43b7-892a-7563782ab3f3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:17.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:17.152971 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98f3db-7088-4a33-8cbd-b958d4e88856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:17.153677 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98f3db-7088-4a33-8cbd-b958d4e88856 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:17.153960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98f3db-7088-4a33-8cbd-b958d4e88856 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:17.154281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98f3db-7088-4a33-8cbd-b958d4e88856 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:17.236057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c98f3db-7088-4a33-8cbd-b958d4e88856 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:17.237778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c98f3db-7088-4a33-8cbd-b958d4e88856 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:17.257761 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2ca32-7b43-43b7-892a-7563782ab3f3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:17.283531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:17.297684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea924a-48fa-4d8b-8dd1-de0a3254458e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:17.299255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea924a-48fa-4d8b-8dd1-de0a3254458e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:17.299645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea924a-48fa-4d8b-8dd1-de0a3254458e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:17.299885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea924a-48fa-4d8b-8dd1-de0a3254458e None admin] PATH_INFO: `/v3/users/442acb89c935401da075802a768b8ca3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:17.301035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea924a-48fa-4d8b-8dd1-de0a3254458e None admin] RBAC: Authorizing `identity:get_user(user_id=442acb89c935401da075802a768b8ca3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:17.302809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea924a-48fa-4d8b-8dd1-de0a3254458e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:17.304448 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 109/223] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:17 2022] GET /identity/v3/users/442acb89c935401da075802a768b8ca3 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:17.305743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccacdf-208e-46c7-a17c-9d0102ca1b5c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:17.309234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccacdf-208e-46c7-a17c-9d0102ca1b5c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:17.309234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccacdf-208e-46c7-a17c-9d0102ca1b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:17.323961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-79ccacdf-208e-46c7-a17c-9d0102ca1b5c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:17.325155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-79ccacdf-208e-46c7-a17c-9d0102ca1b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:17.362418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 115/224] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:17.363302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98f3db-7088-4a33-8cbd-b958d4e88856 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:17.395647 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:17.400897 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 110/225] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:17.407073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ccacdf-208e-46c7-a17c-9d0102ca1b5c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:17.485746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98f3db-7088-4a33-8cbd-b958d4e88856 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:17.524800 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ccacdf-208e-46c7-a17c-9d0102ca1b5c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:17.525954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17db69-3641-4e84-99ed-a8f1d323f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:17.526907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17db69-3641-4e84-99ed-a8f1d323f1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:17.527258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17db69-3641-4e84-99ed-a8f1d323f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:17.527537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17db69-3641-4e84-99ed-a8f1d323f1d8 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:17.528746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17db69-3641-4e84-99ed-a8f1d323f1d8 None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:17.530263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17db69-3641-4e84-99ed-a8f1d323f1d8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:17.532228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 116/226] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:17 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:17.533017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17db69-3641-4e84-99ed-a8f1d323f1d8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:17.542657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:17.569921 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb71e07-b6cf-4c94-a4df-c3dc536d16a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:17.571109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb71e07-b6cf-4c94-a4df-c3dc536d16a4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:17.571852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb71e07-b6cf-4c94-a4df-c3dc536d16a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:17.573198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb71e07-b6cf-4c94-a4df-c3dc536d16a4 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:17.574637 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb71e07-b6cf-4c94-a4df-c3dc536d16a4 None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:17.576301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb71e07-b6cf-4c94-a4df-c3dc536d16a4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:17.578834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 111/227] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:17 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:17.579909 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb71e07-b6cf-4c94-a4df-c3dc536d16a4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:17.651756 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:17.653147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17db69-3641-4e84-99ed-a8f1d323f1d8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:17.686194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb71e07-b6cf-4c94-a4df-c3dc536d16a4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:17.692970 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597e3cc-67dd-4ef3-9468-e55392b5c83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:17.694628 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597e3cc-67dd-4ef3-9468-e55392b5c83e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:17.696395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597e3cc-67dd-4ef3-9468-e55392b5c83e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:17.696395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597e3cc-67dd-4ef3-9468-e55392b5c83e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:17.708104 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597e3cc-67dd-4ef3-9468-e55392b5c83e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b597e3cc-67dd-4ef3-9468-e55392b5c83e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:17.716988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:17.717954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:17.719169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:17.719169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:17.719169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:17.719169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:17.719169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:17.719169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:17.719169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:17.719169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:17.719169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:17.719169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:17.719169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:17.719169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:17.719169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 117/228] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:55:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:17.720086 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597e3cc-67dd-4ef3-9468-e55392b5c83e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:17.727616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03c52c-8ae2-4d56-ac17-dc448513c332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:17.728536 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03c52c-8ae2-4d56-ac17-dc448513c332 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:17.728823 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03c52c-8ae2-4d56-ac17-dc448513c332 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:17.729093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03c52c-8ae2-4d56-ac17-dc448513c332 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0/users/442acb89c935401da075802a768b8ca3/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:17.731132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03c52c-8ae2-4d56-ac17-dc448513c332 None admin] RBAC: Authorizing `identity:create_grant(project_id=62e3ae8270a64809a9778530c2b3b7f0, user_id=442acb89c935401da075802a768b8ca3, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:17.732607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03c52c-8ae2-4d56-ac17-dc448513c332 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:17.748583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 112/229] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:55:17 2022] PUT /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0/users/442acb89c935401da075802a768b8ca3/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:55:17.749731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03c52c-8ae2-4d56-ac17-dc448513c332 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:17.796837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:17.837324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597e3cc-67dd-4ef3-9468-e55392b5c83e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:17.868214 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03c52c-8ae2-4d56-ac17-dc448513c332 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:17.875475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb0391-4d49-4de0-a054-545e73483061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:17.876394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb0391-4d49-4de0-a054-545e73483061 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:17.877096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb0391-4d49-4de0-a054-545e73483061 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:17.877472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb0391-4d49-4de0-a054-545e73483061 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:17.878457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb0391-4d49-4de0-a054-545e73483061 None admin] RBAC: Adding query filter params (role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d, scope.project.id=62e3ae8270a64809a9778530c2b3b7f0, user.id=53aba92d86724ccf876f3f10db8ada11) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:17.878939 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb0391-4d49-4de0-a054-545e73483061 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:17.880596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb0391-4d49-4de0-a054-545e73483061 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:17.887124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 118/230] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:17 2022] GET /identity/v3/role_assignments?user.id=53aba92d86724ccf876f3f10db8ada11&scope.project.id=62e3ae8270a64809a9778530c2b3b7f0&role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:17.907918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:17.922831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e87462-f9f1-43a2-a057-d6f8c0baed2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:17.923532 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e87462-f9f1-43a2-a057-d6f8c0baed2e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:17.923855 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e87462-f9f1-43a2-a057-d6f8c0baed2e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:17.924259 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e87462-f9f1-43a2-a057-d6f8c0baed2e None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:17.925031 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e87462-f9f1-43a2-a057-d6f8c0baed2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:17.925338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e87462-f9f1-43a2-a057-d6f8c0baed2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:17.926853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e87462-f9f1-43a2-a057-d6f8c0baed2e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:17.939929 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 113/231] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:17.945818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb0391-4d49-4de0-a054-545e73483061 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:18.090531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb0391-4d49-4de0-a054-545e73483061 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:18.128370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:18.137253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e023a4-b912-4b8a-86ff-6c5935360f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:18.138372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e023a4-b912-4b8a-86ff-6c5935360f35 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:18.138657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e023a4-b912-4b8a-86ff-6c5935360f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:18.138990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e023a4-b912-4b8a-86ff-6c5935360f35 None admin] PATH_INFO: `/v3/users/442acb89c935401da075802a768b8ca3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:18.140220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e023a4-b912-4b8a-86ff-6c5935360f35 None admin] RBAC: Authorizing `identity:get_user(user_id=442acb89c935401da075802a768b8ca3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:18.141807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e023a4-b912-4b8a-86ff-6c5935360f35 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:18.143766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 119/232] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:17 2022] GET /identity/v3/users/442acb89c935401da075802a768b8ca3 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:18.149717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e87462-f9f1-43a2-a057-d6f8c0baed2e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:18.252362 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e87462-f9f1-43a2-a057-d6f8c0baed2e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:18.285850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a105f9e-be8c-4165-9777-d19c3073d31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:18.286729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a105f9e-be8c-4165-9777-d19c3073d31e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:18.287011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a105f9e-be8c-4165-9777-d19c3073d31e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:18.287291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a105f9e-be8c-4165-9777-d19c3073d31e None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:18.288472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a105f9e-be8c-4165-9777-d19c3073d31e None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:18.289984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a105f9e-be8c-4165-9777-d19c3073d31e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:18.292007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 114/233] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:18 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:18.297943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e023a4-b912-4b8a-86ff-6c5935360f35 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:18.433908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e023a4-b912-4b8a-86ff-6c5935360f35 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:18.483500 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512e7f0-876c-41be-8776-f86c4681b9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:18.484434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512e7f0-876c-41be-8776-f86c4681b9bd None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:18.484717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512e7f0-876c-41be-8776-f86c4681b9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:18.484991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512e7f0-876c-41be-8776-f86c4681b9bd None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0/users/442acb89c935401da075802a768b8ca3/roles/78a15fc7951d4f28beb57f54dace457d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:18.491841 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512e7f0-876c-41be-8776-f86c4681b9bd None admin] RBAC: Authorizing `identity:create_grant(project_id=62e3ae8270a64809a9778530c2b3b7f0, user_id=442acb89c935401da075802a768b8ca3, role_id=78a15fc7951d4f28beb57f54dace457d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:18.493379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512e7f0-876c-41be-8776-f86c4681b9bd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:18.510207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 120/234] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:55:18 2022] PUT /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0/users/442acb89c935401da075802a768b8ca3/roles/78a15fc7951d4f28beb57f54dace457d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:55:19.527549 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b2553-999f-4c67-b9c8-d65f7e2348b0 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:19.528051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b2553-999f-4c67-b9c8-d65f7e2348b0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:19.528125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b2553-999f-4c67-b9c8-d65f7e2348b0 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:19.529772 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 115/235] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:19.535573 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b0eda-1214-4af3-8b6f-ee43c5e471ba None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:19.535990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b0eda-1214-4af3-8b6f-ee43c5e471ba None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:19.536157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b0eda-1214-4af3-8b6f-ee43c5e471ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:19.561530 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-818b0eda-1214-4af3-8b6f-ee43c5e471ba None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:19.562769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-818b0eda-1214-4af3-8b6f-ee43c5e471ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:19.608248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b93094-4af1-4f35-8e0f-ea0ded25d40c None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:19.608587 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b93094-4af1-4f35-8e0f-ea0ded25d40c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:19.608816 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b93094-4af1-4f35-8e0f-ea0ded25d40c None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:19.610576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 116/236] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:19.617010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c094fc-2049-4bdc-99cd-d3ce704cc63b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:19.617304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c094fc-2049-4bdc-99cd-d3ce704cc63b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:19.617591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c094fc-2049-4bdc-99cd-d3ce704cc63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:19.636820 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f9c094fc-2049-4bdc-99cd-d3ce704cc63b None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:19.637641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:19.638825 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f9c094fc-2049-4bdc-99cd-d3ce704cc63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:19.663231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 121/237] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:19.700461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b0eda-1214-4af3-8b6f-ee43c5e471ba None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:19.701074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:19.726736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 117/238] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:19.806032 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c094fc-2049-4bdc-99cd-d3ce704cc63b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:19.859247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b0eda-1214-4af3-8b6f-ee43c5e471ba None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:19.917021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6199ee1-3a73-4411-b20a-5afa58092113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:19.925643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6199ee1-3a73-4411-b20a-5afa58092113 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:19.925643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6199ee1-3a73-4411-b20a-5afa58092113 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:19.925643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6199ee1-3a73-4411-b20a-5afa58092113 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:19.929461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6199ee1-3a73-4411-b20a-5afa58092113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b6199ee1-3a73-4411-b20a-5afa58092113 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:19.936808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:19.937719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:19.938565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:19.938565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:19.938565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:19.938565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:19.938565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:19.938565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:19.938565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:19.938565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:19.938565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:19.938565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:19.938565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:19.938565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:19.938565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 122/239] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:19.945989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6199ee1-3a73-4411-b20a-5afa58092113 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:19.979293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c094fc-2049-4bdc-99cd-d3ce704cc63b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:19.996926 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:20.040695 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ed79c-ea1b-499a-a4e3-3c4d5d5814fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:20.041938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ed79c-ea1b-499a-a4e3-3c4d5d5814fa None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:20.042455 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ed79c-ea1b-499a-a4e3-3c4d5d5814fa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:20.042920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ed79c-ea1b-499a-a4e3-3c4d5d5814fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:20.050405 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ed79c-ea1b-499a-a4e3-3c4d5d5814fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:20.052198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ed79c-ea1b-499a-a4e3-3c4d5d5814fa None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:20.054163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 118/240] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:20.055289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6199ee1-3a73-4411-b20a-5afa58092113 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:20.060057 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ed79c-ea1b-499a-a4e3-3c4d5d5814fa None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:20.116377 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dae81c-3a7a-4c77-9e76-8cc4cfe4424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:20.116919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dae81c-3a7a-4c77-9e76-8cc4cfe4424d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:20.117210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dae81c-3a7a-4c77-9e76-8cc4cfe4424d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:20.117487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dae81c-3a7a-4c77-9e76-8cc4cfe4424d None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:20.118356 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dae81c-3a7a-4c77-9e76-8cc4cfe4424d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:20.118607 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dae81c-3a7a-4c77-9e76-8cc4cfe4424d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:20.120018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dae81c-3a7a-4c77-9e76-8cc4cfe4424d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:20.138490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 123/241] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:20.142260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:20.144487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dae81c-3a7a-4c77-9e76-8cc4cfe4424d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:20.192569 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ed79c-ea1b-499a-a4e3-3c4d5d5814fa None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:20.238437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1cab6-0b7e-47f8-938d-1a4a3c9984b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:20.239573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1cab6-0b7e-47f8-938d-1a4a3c9984b8 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:20.240051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1cab6-0b7e-47f8-938d-1a4a3c9984b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:20.240532 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1cab6-0b7e-47f8-938d-1a4a3c9984b8 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:20.241456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1cab6-0b7e-47f8-938d-1a4a3c9984b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:20.243139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1cab6-0b7e-47f8-938d-1a4a3c9984b8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:20.289650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dae81c-3a7a-4c77-9e76-8cc4cfe4424d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:20.292491 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 119/242] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:55:20 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:55:20.300320 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:20.327270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ee7a7-fc67-4ec2-b721-5a1113705abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:20.328083 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee7a7-fc67-4ec2-b721-5a1113705abd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:20.328372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee7a7-fc67-4ec2-b721-5a1113705abd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:20.328654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee7a7-fc67-4ec2-b721-5a1113705abd None admin] PATH_INFO: `/v3/users/341b5c5a8b3e4662b9a08e0617382827` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:20.345943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ee7a7-fc67-4ec2-b721-5a1113705abd None admin] RBAC: Authorizing `identity:get_user(user_id=341b5c5a8b3e4662b9a08e0617382827)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:20.347431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ee7a7-fc67-4ec2-b721-5a1113705abd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:20.349222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 124/243] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:20 2022] GET /identity/v3/users/341b5c5a8b3e4662b9a08e0617382827 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:20.355433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1cab6-0b7e-47f8-938d-1a4a3c9984b8 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:20.477579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1cab6-0b7e-47f8-938d-1a4a3c9984b8 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:20.493047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203dfdf0-4e2d-4610-b69f-0648db5253cf None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:20.493364 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203dfdf0-4e2d-4610-b69f-0648db5253cf None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:20.493650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203dfdf0-4e2d-4610-b69f-0648db5253cf None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:20.495083 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 125/244] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:20.499228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:20.503089 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5567c88-ee24-45e3-965d-1a8895276452 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:20.503387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5567c88-ee24-45e3-965d-1a8895276452 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:20.503660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5567c88-ee24-45e3-965d-1a8895276452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:20.530146 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f5567c88-ee24-45e3-965d-1a8895276452 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:20.531377 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f5567c88-ee24-45e3-965d-1a8895276452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:20.551980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df865400-f0ec-488a-97cc-8c272a658b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:20.552646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df865400-f0ec-488a-97cc-8c272a658b57 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:20.552933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df865400-f0ec-488a-97cc-8c272a658b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:20.553214 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df865400-f0ec-488a-97cc-8c272a658b57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:20.612165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 126/245] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:20.636883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5567c88-ee24-45e3-965d-1a8895276452 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:20.641709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df865400-f0ec-488a-97cc-8c272a658b57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:20.643957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df865400-f0ec-488a-97cc-8c272a658b57 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:20.738839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:20.742868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5567c88-ee24-45e3-965d-1a8895276452 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:20.764256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 120/246] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:20.771077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df865400-f0ec-488a-97cc-8c272a658b57 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:20.797065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:20.798212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3af9a6-571d-474d-ac92-8ca6a7602c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:20.799264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3af9a6-571d-474d-ac92-8ca6a7602c27 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:20.799553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3af9a6-571d-474d-ac92-8ca6a7602c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:20.799837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3af9a6-571d-474d-ac92-8ca6a7602c27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:20.819112 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3af9a6-571d-474d-ac92-8ca6a7602c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-bf3af9a6-571d-474d-ac92-8ca6a7602c27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:20.828203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:20.829372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:20.830277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:20.830277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:20.830277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:20.830277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:20.830277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:20.830277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:20.830277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:20.830277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:20.830277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:20.830277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:20.830277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:20.830277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:20.834077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 127/247] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:55:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:20.836066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3af9a6-571d-474d-ac92-8ca6a7602c27 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:20.898955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df865400-f0ec-488a-97cc-8c272a658b57 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:20.939783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25354848-c02d-4357-859c-90c27b94831d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:20.941004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25354848-c02d-4357-859c-90c27b94831d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:20.941510 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25354848-c02d-4357-859c-90c27b94831d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:20.941992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25354848-c02d-4357-859c-90c27b94831d None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:20.943439 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25354848-c02d-4357-859c-90c27b94831d None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:20.945099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25354848-c02d-4357-859c-90c27b94831d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:20.947352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 121/248] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:20 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:20.948773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76212c71-fac5-43cc-bfa9-3ef59c8d9538 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:20.949227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76212c71-fac5-43cc-bfa9-3ef59c8d9538 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:20.949731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76212c71-fac5-43cc-bfa9-3ef59c8d9538 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:20.951409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 122/249] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:20.953802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3af9a6-571d-474d-ac92-8ca6a7602c27 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:20.957655 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c58941-8a26-48c8-8a23-cc727e5d85c9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:20.958053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c58941-8a26-48c8-8a23-cc727e5d85c9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:20.958382 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c58941-8a26-48c8-8a23-cc727e5d85c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:20.977808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-43c58941-8a26-48c8-8a23-cc727e5d85c9 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:20.979081 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-43c58941-8a26-48c8-8a23-cc727e5d85c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:20.993077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f1fa7-eca0-4af9-a484-49bddb1f9c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:20.993922 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f1fa7-eca0-4af9-a484-49bddb1f9c0d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:20.994216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f1fa7-eca0-4af9-a484-49bddb1f9c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:20.995157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f1fa7-eca0-4af9-a484-49bddb1f9c0d None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:20.995336 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f1fa7-eca0-4af9-a484-49bddb1f9c0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:20.995666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f1fa7-eca0-4af9-a484-49bddb1f9c0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:20.997120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f1fa7-eca0-4af9-a484-49bddb1f9c0d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:21.010842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 128/250] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:21.011825 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f1fa7-eca0-4af9-a484-49bddb1f9c0d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:21.040666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:21.061920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 123/251] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:21.062821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c58941-8a26-48c8-8a23-cc727e5d85c9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:21.081984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:21.124219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f1fa7-eca0-4af9-a484-49bddb1f9c0d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:21.160638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94116d6-7f7a-4df2-9cf8-4124428a02af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:21.161222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94116d6-7f7a-4df2-9cf8-4124428a02af None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:21.161510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94116d6-7f7a-4df2-9cf8-4124428a02af None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:21.161820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94116d6-7f7a-4df2-9cf8-4124428a02af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:21.162665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94116d6-7f7a-4df2-9cf8-4124428a02af None admin] RBAC: Adding query filter params (role.id=f420d4b9bfb64b0f8cd462e9d72bd24d, scope.project.id=62e3ae8270a64809a9778530c2b3b7f0, user.id=341b5c5a8b3e4662b9a08e0617382827) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:21.162981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94116d6-7f7a-4df2-9cf8-4124428a02af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:21.164468 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94116d6-7f7a-4df2-9cf8-4124428a02af None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:21.170662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 129/252] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:21 2022] GET /identity/v3/role_assignments?user.id=341b5c5a8b3e4662b9a08e0617382827&scope.project.id=62e3ae8270a64809a9778530c2b3b7f0&role.id=f420d4b9bfb64b0f8cd462e9d72bd24d => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:21.179657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94116d6-7f7a-4df2-9cf8-4124428a02af None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:21.246226 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c58941-8a26-48c8-8a23-cc727e5d85c9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:21.279469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:21.304099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e43201-f340-477e-b79d-7c42b8619655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:21.305002 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e43201-f340-477e-b79d-7c42b8619655 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:21.305284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e43201-f340-477e-b79d-7c42b8619655 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:21.305478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94116d6-7f7a-4df2-9cf8-4124428a02af None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:21.305577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e43201-f340-477e-b79d-7c42b8619655 None admin] PATH_INFO: `/v3/users/442acb89c935401da075802a768b8ca3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:21.306827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e43201-f340-477e-b79d-7c42b8619655 None admin] RBAC: Authorizing `identity:get_user(user_id=442acb89c935401da075802a768b8ca3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:21.308314 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e43201-f340-477e-b79d-7c42b8619655 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:21.310015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 124/253] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:21 2022] GET /identity/v3/users/442acb89c935401da075802a768b8ca3 => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:21.311222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512a824-7d25-4fd2-917a-80eb8f2e1d5d None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:21.311507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512a824-7d25-4fd2-917a-80eb8f2e1d5d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:21.311781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512a824-7d25-4fd2-917a-80eb8f2e1d5d None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:21.313008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 125/254] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:21.323245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305a3c6-0fb1-4441-8730-84bc8d514622 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:21.323529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305a3c6-0fb1-4441-8730-84bc8d514622 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:21.323814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305a3c6-0fb1-4441-8730-84bc8d514622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:21.358443 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-8305a3c6-0fb1-4441-8730-84bc8d514622 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:21.359682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8305a3c6-0fb1-4441-8730-84bc8d514622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:21.363167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fdfd6-20d0-43fd-98f4-b8c9bb002622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:21.364001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdfd6-20d0-43fd-98f4-b8c9bb002622 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:21.364292 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdfd6-20d0-43fd-98f4-b8c9bb002622 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:21.364573 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdfd6-20d0-43fd-98f4-b8c9bb002622 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:21.387925 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fdfd6-20d0-43fd-98f4-b8c9bb002622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-2b5fdfd6-20d0-43fd-98f4-b8c9bb002622 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:21.397186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:21.398169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:21.398940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:21.398940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:21.398940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:21.398940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:21.398940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:21.398940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:21.398940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:21.398940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:21.398940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:21.398940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:21.398940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:21.398940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:21.402676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 130/255] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:21.403596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fdfd6-20d0-43fd-98f4-b8c9bb002622 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:21.497241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 126/256] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:21.498334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305a3c6-0fb1-4441-8730-84bc8d514622 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:21.520774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:21.537232 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fdfd6-20d0-43fd-98f4-b8c9bb002622 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:21.592198 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35631b2a-91eb-4d0f-8797-3d741b34289c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:21.592868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35631b2a-91eb-4d0f-8797-3d741b34289c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:21.593155 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35631b2a-91eb-4d0f-8797-3d741b34289c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:21.593441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35631b2a-91eb-4d0f-8797-3d741b34289c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:21.626065 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:21.685128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305a3c6-0fb1-4441-8730-84bc8d514622 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:21.722576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd707f04-ad3d-4f03-b09d-da541b8ab1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:21.723257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd707f04-ad3d-4f03-b09d-da541b8ab1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:21.724148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd707f04-ad3d-4f03-b09d-da541b8ab1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:21.724426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd707f04-ad3d-4f03-b09d-da541b8ab1b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:21.725327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd707f04-ad3d-4f03-b09d-da541b8ab1b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:21.725986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd707f04-ad3d-4f03-b09d-da541b8ab1b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:21.727396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd707f04-ad3d-4f03-b09d-da541b8ab1b9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:21.729776 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35631b2a-91eb-4d0f-8797-3d741b34289c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:21.731338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35631b2a-91eb-4d0f-8797-3d741b34289c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:21.736607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 127/257] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:21.737434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd707f04-ad3d-4f03-b09d-da541b8ab1b9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:21.853366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd707f04-ad3d-4f03-b09d-da541b8ab1b9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:21.878044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:21.902641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81514a-bff4-48a6-8b31-03b07db21876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:21.903471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81514a-bff4-48a6-8b31-03b07db21876 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:21.903753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81514a-bff4-48a6-8b31-03b07db21876 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:21.904026 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81514a-bff4-48a6-8b31-03b07db21876 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:21.912140 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 131/258] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:21.913008 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35631b2a-91eb-4d0f-8797-3d741b34289c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:21.928715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81514a-bff4-48a6-8b31-03b07db21876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8a81514a-bff4-48a6-8b31-03b07db21876 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:21.936842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:21.937738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:21.938645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:21.938645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:21.938645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:21.938645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:21.938645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:21.938645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:21.938645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:21.938645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:21.938645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:21.938645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:21.938645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:21.938645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:21.946375 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 128/259] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:55:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:21.947052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81514a-bff4-48a6-8b31-03b07db21876 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:21.968203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:22.046487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35631b2a-91eb-4d0f-8797-3d741b34289c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:22.109742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2dd20-9aa8-4f40-9ce7-5a71b52129b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:22.111143 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2dd20-9aa8-4f40-9ce7-5a71b52129b1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:22.111447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2dd20-9aa8-4f40-9ce7-5a71b52129b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:22.111727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2dd20-9aa8-4f40-9ce7-5a71b52129b1 None admin] PATH_INFO: `/v3/users/53aba92d86724ccf876f3f10db8ada11` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:22.113205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2dd20-9aa8-4f40-9ce7-5a71b52129b1 None admin] RBAC: Authorizing `identity:get_user(user_id=53aba92d86724ccf876f3f10db8ada11)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:22.114738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2dd20-9aa8-4f40-9ce7-5a71b52129b1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:22.119492 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 132/260] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:21 2022] GET /identity/v3/users/53aba92d86724ccf876f3f10db8ada11 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:22.135541 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2dd20-9aa8-4f40-9ce7-5a71b52129b1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:22.157672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81514a-bff4-48a6-8b31-03b07db21876 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:22.228270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:22.239890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc9c56-0008-4196-aeb2-d6ea45a73c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:22.240759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc9c56-0008-4196-aeb2-d6ea45a73c3f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:22.241038 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc9c56-0008-4196-aeb2-d6ea45a73c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:22.241312 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc9c56-0008-4196-aeb2-d6ea45a73c3f None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:22.243431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc9c56-0008-4196-aeb2-d6ea45a73c3f None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:22.244903 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc9c56-0008-4196-aeb2-d6ea45a73c3f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:22.246803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 129/261] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:21 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:22.247382 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc9c56-0008-4196-aeb2-d6ea45a73c3f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:22.300931 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2dd20-9aa8-4f40-9ce7-5a71b52129b1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:22.328146 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:22.363650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6862b-ab4c-45a9-8a17-91e9ff142d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:22.364241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6862b-ab4c-45a9-8a17-91e9ff142d2d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:22.364518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6862b-ab4c-45a9-8a17-91e9ff142d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:22.364998 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6862b-ab4c-45a9-8a17-91e9ff142d2d None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:22.366881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6862b-ab4c-45a9-8a17-91e9ff142d2d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:22.367199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6862b-ab4c-45a9-8a17-91e9ff142d2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:22.368591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6862b-ab4c-45a9-8a17-91e9ff142d2d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:22.388789 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 133/262] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:22.389612 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6862b-ab4c-45a9-8a17-91e9ff142d2d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:22.439634 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc9c56-0008-4196-aeb2-d6ea45a73c3f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:22.506525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963da8d-b44b-41cf-8cb9-d1168f7608d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:22.507403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963da8d-b44b-41cf-8cb9-d1168f7608d8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:22.507681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963da8d-b44b-41cf-8cb9-d1168f7608d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:22.507954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963da8d-b44b-41cf-8cb9-d1168f7608d8 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:22.514533 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963da8d-b44b-41cf-8cb9-d1168f7608d8 None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:22.516013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963da8d-b44b-41cf-8cb9-d1168f7608d8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:22.517901 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 130/263] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:22 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:22.521894 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963da8d-b44b-41cf-8cb9-d1168f7608d8 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:22.591767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6862b-ab4c-45a9-8a17-91e9ff142d2d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:22.617218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:22.658003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963da8d-b44b-41cf-8cb9-d1168f7608d8 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:22.664170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadd4e1-ff18-4d79-887c-540816e89c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:22.664844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadd4e1-ff18-4d79-887c-540816e89c13 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:22.665134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadd4e1-ff18-4d79-887c-540816e89c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:22.665415 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadd4e1-ff18-4d79-887c-540816e89c13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:22.666330 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eadd4e1-ff18-4d79-887c-540816e89c13 None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=62e3ae8270a64809a9778530c2b3b7f0, user.id=442acb89c935401da075802a768b8ca3) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:22.666644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eadd4e1-ff18-4d79-887c-540816e89c13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:22.668127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eadd4e1-ff18-4d79-887c-540816e89c13 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:22.677518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 134/264] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:22 2022] GET /identity/v3/role_assignments?user.id=442acb89c935401da075802a768b8ca3&scope.project.id=62e3ae8270a64809a9778530c2b3b7f0&role.id=a3de9df253294d069dc1867fa26600c3 => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:22.678453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadd4e1-ff18-4d79-887c-540816e89c13 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:22.715320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c05a5-ceae-4550-9f56-5d6eea97f911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:22.716386 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c05a5-ceae-4550-9f56-5d6eea97f911 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:22.716677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c05a5-ceae-4550-9f56-5d6eea97f911 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:22.716963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c05a5-ceae-4550-9f56-5d6eea97f911 None admin] PATH_INFO: `/v3/users/442acb89c935401da075802a768b8ca3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:22.718464 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c05a5-ceae-4550-9f56-5d6eea97f911 None admin] RBAC: Authorizing `identity:get_user(user_id=442acb89c935401da075802a768b8ca3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:22.720433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c05a5-ceae-4550-9f56-5d6eea97f911 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:22.723086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 131/265] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:22 2022] GET /identity/v3/users/442acb89c935401da075802a768b8ca3 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:22.734785 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c05a5-ceae-4550-9f56-5d6eea97f911 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:22.748356 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:22.814572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadd4e1-ff18-4d79-887c-540816e89c13 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:22.883573 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92abfa-d437-4f2b-9df0-705c847be1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:22.884402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92abfa-d437-4f2b-9df0-705c847be1de None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:22.884686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92abfa-d437-4f2b-9df0-705c847be1de None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:22.884962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92abfa-d437-4f2b-9df0-705c847be1de None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0/users/53aba92d86724ccf876f3f10db8ada11/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:22.899592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92abfa-d437-4f2b-9df0-705c847be1de None admin] RBAC: Authorizing `identity:create_grant(project_id=62e3ae8270a64809a9778530c2b3b7f0, user_id=53aba92d86724ccf876f3f10db8ada11, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:22.901065 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92abfa-d437-4f2b-9df0-705c847be1de None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:22.901277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c05a5-ceae-4550-9f56-5d6eea97f911 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:22.932835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 135/266] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:55:22 2022] PUT /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0/users/53aba92d86724ccf876f3f10db8ada11/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:55:22.965244 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4aa5a-547d-45e1-a80e-57c5e1e07b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:22.966379 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4aa5a-547d-45e1-a80e-57c5e1e07b87 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:22.966672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4aa5a-547d-45e1-a80e-57c5e1e07b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:22.966957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4aa5a-547d-45e1-a80e-57c5e1e07b87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:22.973323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:23.064683 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4aa5a-547d-45e1-a80e-57c5e1e07b87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:23.066304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4aa5a-547d-45e1-a80e-57c5e1e07b87 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:23.182677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 132/267] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:23.187958 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92abfa-d437-4f2b-9df0-705c847be1de None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:23.252709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3d2f6-2be9-4850-b028-aaacbf77cea7 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:23.253375 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3d2f6-2be9-4850-b028-aaacbf77cea7 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:23.253934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3d2f6-2be9-4850-b028-aaacbf77cea7 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:23.255532 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 133/268] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:23.261604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cbd82-1e60-4f64-8b6e-6a4f737d249e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:23.262162 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cbd82-1e60-4f64-8b6e-6a4f737d249e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:23.262638 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cbd82-1e60-4f64-8b6e-6a4f737d249e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:23.284494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7a4cbd82-1e60-4f64-8b6e-6a4f737d249e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:23.286082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7a4cbd82-1e60-4f64-8b6e-6a4f737d249e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:23.300973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:23.305736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92abfa-d437-4f2b-9df0-705c847be1de None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:23.320735 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:23.348255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc22c2b-8b86-465f-b926-04ada01886a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:23.349141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc22c2b-8b86-465f-b926-04ada01886a5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:23.349430 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc22c2b-8b86-465f-b926-04ada01886a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:23.351024 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc22c2b-8b86-465f-b926-04ada01886a5 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:23.354036 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc22c2b-8b86-465f-b926-04ada01886a5 None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:23.354847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc22c2b-8b86-465f-b926-04ada01886a5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:23.357000 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 136/269] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:23 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:23.358349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f3774-3d0c-4dad-9f1f-4311fcc02468 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:23.358644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f3774-3d0c-4dad-9f1f-4311fcc02468 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:23.358926 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f3774-3d0c-4dad-9f1f-4311fcc02468 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:23.360562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 137/270] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:23.365500 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f3774-3d0c-4dad-9f1f-4311fcc02468 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:23.370409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 134/271] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:23.371739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48776d5-937d-4f7c-b50e-1ec9935e7d85 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:23.372026 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48776d5-937d-4f7c-b50e-1ec9935e7d85 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:23.372300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48776d5-937d-4f7c-b50e-1ec9935e7d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:23.390785 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e48776d5-937d-4f7c-b50e-1ec9935e7d85 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:23.392172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e48776d5-937d-4f7c-b50e-1ec9935e7d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:23.476984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 135/272] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:23.477728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48776d5-937d-4f7c-b50e-1ec9935e7d85 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:23.516301 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f3774-3d0c-4dad-9f1f-4311fcc02468 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:23.560236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323363a2-bc91-4e02-be04-be8b82952387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:23.561422 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323363a2-bc91-4e02-be04-be8b82952387 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:23.561751 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323363a2-bc91-4e02-be04-be8b82952387 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:23.562308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323363a2-bc91-4e02-be04-be8b82952387 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:23.563163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323363a2-bc91-4e02-be04-be8b82952387 None admin] RBAC: Adding query filter params (role.id=78a15fc7951d4f28beb57f54dace457d, scope.project.id=62e3ae8270a64809a9778530c2b3b7f0, user.id=442acb89c935401da075802a768b8ca3) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:23.563661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323363a2-bc91-4e02-be04-be8b82952387 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:23.565978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323363a2-bc91-4e02-be04-be8b82952387 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:23.573450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 138/273] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:23 2022] GET /identity/v3/role_assignments?user.id=442acb89c935401da075802a768b8ca3&scope.project.id=62e3ae8270a64809a9778530c2b3b7f0&role.id=78a15fc7951d4f28beb57f54dace457d => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:23.574269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323363a2-bc91-4e02-be04-be8b82952387 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:23.618018 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:23.629045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48776d5-937d-4f7c-b50e-1ec9935e7d85 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:23.648715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:23.690746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec52ffa-fd66-468f-a451-9bb24f469421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:23.691845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec52ffa-fd66-468f-a451-9bb24f469421 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:23.692316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec52ffa-fd66-468f-a451-9bb24f469421 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:23.692784 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec52ffa-fd66-468f-a451-9bb24f469421 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:23.694228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec52ffa-fd66-468f-a451-9bb24f469421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:23.695919 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec52ffa-fd66-468f-a451-9bb24f469421 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:23.698043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 136/274] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:23.704292 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec52ffa-fd66-468f-a451-9bb24f469421 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:23.789946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323363a2-bc91-4e02-be04-be8b82952387 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:23.813494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec52ffa-fd66-468f-a451-9bb24f469421 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:23.842072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab657e-b206-4b09-a54b-07eac4d417c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:23.843134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab657e-b206-4b09-a54b-07eac4d417c0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:23.843414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab657e-b206-4b09-a54b-07eac4d417c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:23.843682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab657e-b206-4b09-a54b-07eac4d417c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:23.860291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab657e-b206-4b09-a54b-07eac4d417c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b9ab657e-b206-4b09-a54b-07eac4d417c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:23.870995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:23.871891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:23.872692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:23.872692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:23.872692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:23.872692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:23.872692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:23.872692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:23.872692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:23.872692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:23.872692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:23.872692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:23.872692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:23.872692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:23.872692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 139/275] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:23.879202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab657e-b206-4b09-a54b-07eac4d417c0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:23.882890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292273e8-a47f-4ff3-ae32-b37fe8a80536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:23.883705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292273e8-a47f-4ff3-ae32-b37fe8a80536 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:23.884086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292273e8-a47f-4ff3-ae32-b37fe8a80536 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:23.884366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292273e8-a47f-4ff3-ae32-b37fe8a80536 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:23.885114 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292273e8-a47f-4ff3-ae32-b37fe8a80536 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:23.886634 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292273e8-a47f-4ff3-ae32-b37fe8a80536 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:23.915285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 137/276] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:55:23 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:55:23.965422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef6048-7673-4e0e-b409-c658b4f1695b None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:23.966079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef6048-7673-4e0e-b409-c658b4f1695b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:23.966542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef6048-7673-4e0e-b409-c658b4f1695b None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:23.968137 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 138/277] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:23.978590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aba10d-8c87-4b28-8df5-a24c75d045e4 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:23.979208 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aba10d-8c87-4b28-8df5-a24c75d045e4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:23.979704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aba10d-8c87-4b28-8df5-a24c75d045e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:23.995211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:24.001921 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-43aba10d-8c87-4b28-8df5-a24c75d045e4 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:24.004570 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-43aba10d-8c87-4b28-8df5-a24c75d045e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:24.006903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab657e-b206-4b09-a54b-07eac4d417c0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:24.023228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:24.051205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238c1b4-2961-40ed-9e97-6fe0538ee171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:24.051906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238c1b4-2961-40ed-9e97-6fe0538ee171 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:24.052258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238c1b4-2961-40ed-9e97-6fe0538ee171 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:24.052520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238c1b4-2961-40ed-9e97-6fe0538ee171 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:24.053294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238c1b4-2961-40ed-9e97-6fe0538ee171 None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:24.053600 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238c1b4-2961-40ed-9e97-6fe0538ee171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:24.055063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238c1b4-2961-40ed-9e97-6fe0538ee171 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:24.065658 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 140/278] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:24.072640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238c1b4-2961-40ed-9e97-6fe0538ee171 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:24.111427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 139/279] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:24.136929 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aba10d-8c87-4b28-8df5-a24c75d045e4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:24.215537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238c1b4-2961-40ed-9e97-6fe0538ee171 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:24.250914 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aba10d-8c87-4b28-8df5-a24c75d045e4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:24.254962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f0125-413d-46ce-9652-7ae03bdc598c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:24.255964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f0125-413d-46ce-9652-7ae03bdc598c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:24.256342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f0125-413d-46ce-9652-7ae03bdc598c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:24.256712 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f0125-413d-46ce-9652-7ae03bdc598c None admin] PATH_INFO: `/v3/users/06a5910838584896aa26ebcb0ff40d28` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:24.272382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f0125-413d-46ce-9652-7ae03bdc598c None admin] RBAC: Authorizing `identity:get_user(user_id=06a5910838584896aa26ebcb0ff40d28)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:24.274045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f0125-413d-46ce-9652-7ae03bdc598c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:24.276212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 141/280] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:24 2022] GET /identity/v3/users/06a5910838584896aa26ebcb0ff40d28 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:24.284001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f0125-413d-46ce-9652-7ae03bdc598c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:24.290149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d4089-1d62-45a3-bbad-3ae481184878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:24.290958 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d4089-1d62-45a3-bbad-3ae481184878 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:24.291265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d4089-1d62-45a3-bbad-3ae481184878 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:24.291538 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d4089-1d62-45a3-bbad-3ae481184878 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:24.296802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:24.308751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d4089-1d62-45a3-bbad-3ae481184878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-e16d4089-1d62-45a3-bbad-3ae481184878 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:24.317553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:24.318469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:24.319245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:24.319245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:24.319245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:24.319245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:24.319245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:24.319245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:24.319245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:24.319245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:24.319245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:24.319245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:24.319245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:24.319245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:24.319245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 140/281] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:24.326368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d4089-1d62-45a3-bbad-3ae481184878 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:24.344050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:24.415828 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f0125-413d-46ce-9652-7ae03bdc598c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:24.459426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bfbf8-409d-4996-a7fc-d3d1ed3cc16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:24.470419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d4089-1d62-45a3-bbad-3ae481184878 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:24.472655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bfbf8-409d-4996-a7fc-d3d1ed3cc16a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:24.472655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bfbf8-409d-4996-a7fc-d3d1ed3cc16a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:24.472655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bfbf8-409d-4996-a7fc-d3d1ed3cc16a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:24.473801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bfbf8-409d-4996-a7fc-d3d1ed3cc16a None admin] RBAC: Adding query filter params (role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d, user.id=06a5910838584896aa26ebcb0ff40d28, scope.system=all) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:24.474338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bfbf8-409d-4996-a7fc-d3d1ed3cc16a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:24.476751 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bfbf8-409d-4996-a7fc-d3d1ed3cc16a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:24.484461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 142/282] 10.222.0.79 () {56 vars in 1308 bytes} [Mon Jul 4 09:55:24 2022] GET /identity/v3/role_assignments?user.id=06a5910838584896aa26ebcb0ff40d28&scope.system=all&role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:24.500185 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea6218-fe38-4159-84d8-8aa9207a25d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:24.501013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea6218-fe38-4159-84d8-8aa9207a25d4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:24.501246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea6218-fe38-4159-84d8-8aa9207a25d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:24.502031 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea6218-fe38-4159-84d8-8aa9207a25d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:24.502506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea6218-fe38-4159-84d8-8aa9207a25d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:24.502822 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea6218-fe38-4159-84d8-8aa9207a25d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:24.504286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea6218-fe38-4159-84d8-8aa9207a25d4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:24.514517 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 141/283] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:24.520713 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bfbf8-409d-4996-a7fc-d3d1ed3cc16a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:24.609278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:24.640056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bfbf8-409d-4996-a7fc-d3d1ed3cc16a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:24.678723 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea012d71-9603-4561-86c1-e30e4862eaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:24.680007 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea012d71-9603-4561-86c1-e30e4862eaf3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:24.680484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea012d71-9603-4561-86c1-e30e4862eaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:24.680948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea012d71-9603-4561-86c1-e30e4862eaf3 None admin] PATH_INFO: `/v3/users/341b5c5a8b3e4662b9a08e0617382827` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:24.682496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea012d71-9603-4561-86c1-e30e4862eaf3 None admin] RBAC: Authorizing `identity:get_user(user_id=341b5c5a8b3e4662b9a08e0617382827)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:24.684171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea012d71-9603-4561-86c1-e30e4862eaf3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:24.686510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 143/284] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:24 2022] GET /identity/v3/users/341b5c5a8b3e4662b9a08e0617382827 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:24.693669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea6218-fe38-4159-84d8-8aa9207a25d4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:24.777307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:24.812503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea6218-fe38-4159-84d8-8aa9207a25d4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:24.849419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b876a-14bd-49e9-8023-3e66679cf86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:24.850318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b876a-14bd-49e9-8023-3e66679cf86b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:24.850600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b876a-14bd-49e9-8023-3e66679cf86b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:24.851202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b876a-14bd-49e9-8023-3e66679cf86b None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:24.852395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b876a-14bd-49e9-8023-3e66679cf86b None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:24.853911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b876a-14bd-49e9-8023-3e66679cf86b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:24.855625 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 142/285] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:24 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:24.861649 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea012d71-9603-4561-86c1-e30e4862eaf3 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:24.971656 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea012d71-9603-4561-86c1-e30e4862eaf3 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:25.007748 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f20ac1-863a-49b8-8194-5e6a71aee2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:25.008661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f20ac1-863a-49b8-8194-5e6a71aee2aa None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:25.008941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f20ac1-863a-49b8-8194-5e6a71aee2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:25.009215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f20ac1-863a-49b8-8194-5e6a71aee2aa None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0/users/341b5c5a8b3e4662b9a08e0617382827/roles/f420d4b9bfb64b0f8cd462e9d72bd24d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:25.011202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f20ac1-863a-49b8-8194-5e6a71aee2aa None admin] RBAC: Authorizing `identity:create_grant(project_id=62e3ae8270a64809a9778530c2b3b7f0, user_id=341b5c5a8b3e4662b9a08e0617382827, role_id=f420d4b9bfb64b0f8cd462e9d72bd24d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:25.012653 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f20ac1-863a-49b8-8194-5e6a71aee2aa None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:25.030735 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 144/286] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:55:24 2022] PUT /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0/users/341b5c5a8b3e4662b9a08e0617382827/roles/f420d4b9bfb64b0f8cd462e9d72bd24d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:55:25.642541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d6f7-b09c-4d73-83cf-e279388a4024 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:25.642541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d6f7-b09c-4d73-83cf-e279388a4024 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:25.642854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d6f7-b09c-4d73-83cf-e279388a4024 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:25.644525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 143/287] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:25.655802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e8003-bd68-4a85-bf43-dfb1eb662dff None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:25.656131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e8003-bd68-4a85-bf43-dfb1eb662dff None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:25.656358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e8003-bd68-4a85-bf43-dfb1eb662dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:25.661972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:25.678660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ac2e8003-bd68-4a85-bf43-dfb1eb662dff None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:25.679865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ac2e8003-bd68-4a85-bf43-dfb1eb662dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:25.766119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 145/288] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:25.804431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18d6f7-b09c-4d73-83cf-e279388a4024 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:25.911824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18d6f7-b09c-4d73-83cf-e279388a4024 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:25.944687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a31b6d-85a9-4563-ae2f-70c65f4751ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:25.945540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a31b6d-85a9-4563-ae2f-70c65f4751ba None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:25.945853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a31b6d-85a9-4563-ae2f-70c65f4751ba None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:25.946120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a31b6d-85a9-4563-ae2f-70c65f4751ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:25.953042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:25.957002 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a31b6d-85a9-4563-ae2f-70c65f4751ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-25a31b6d-85a9-4563-ae2f-70c65f4751ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:25.964505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:25.965360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:25.966195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:25.966195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:25.966195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:25.966195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:25.966195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:25.966195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:25.966195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:25.966195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:25.966195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:25.966195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:25.966195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:25.966195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:25.966195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 144/289] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:25.972251 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e8003-bd68-4a85-bf43-dfb1eb662dff None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:26.096891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e8003-bd68-4a85-bf43-dfb1eb662dff None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:26.129441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:26.136958 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564da3b-172d-4724-9fd9-382f205e2689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:26.137667 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564da3b-172d-4724-9fd9-382f205e2689 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:26.138072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564da3b-172d-4724-9fd9-382f205e2689 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:26.138364 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564da3b-172d-4724-9fd9-382f205e2689 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:26.139127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564da3b-172d-4724-9fd9-382f205e2689 None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:26.139439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564da3b-172d-4724-9fd9-382f205e2689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:26.140851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564da3b-172d-4724-9fd9-382f205e2689 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:26.153888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 146/290] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:26.166833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a31b6d-85a9-4563-ae2f-70c65f4751ba None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:26.337604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a31b6d-85a9-4563-ae2f-70c65f4751ba None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:26.377601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f835a-0391-4051-a83e-a6136fdcd4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:26.378499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f835a-0391-4051-a83e-a6136fdcd4da None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:26.378776 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f835a-0391-4051-a83e-a6136fdcd4da None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:26.379049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f835a-0391-4051-a83e-a6136fdcd4da None admin] PATH_INFO: `/v3/users/53aba92d86724ccf876f3f10db8ada11` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:26.380248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f835a-0391-4051-a83e-a6136fdcd4da None admin] RBAC: Authorizing `identity:get_user(user_id=53aba92d86724ccf876f3f10db8ada11)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:26.381726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f835a-0391-4051-a83e-a6136fdcd4da None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:26.383650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 145/291] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:26 2022] GET /identity/v3/users/53aba92d86724ccf876f3f10db8ada11 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:26.389055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564da3b-172d-4724-9fd9-382f205e2689 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:26.520115 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564da3b-172d-4724-9fd9-382f205e2689 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:26.561065 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348a3bd-2493-41db-ac17-75e18c4bff14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:26.561760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348a3bd-2493-41db-ac17-75e18c4bff14 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:26.562046 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348a3bd-2493-41db-ac17-75e18c4bff14 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:26.562325 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348a3bd-2493-41db-ac17-75e18c4bff14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:26.642282 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:26.647686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348a3bd-2493-41db-ac17-75e18c4bff14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:26.649183 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348a3bd-2493-41db-ac17-75e18c4bff14 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:26.754510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 147/292] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:26.760064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f835a-0391-4051-a83e-a6136fdcd4da None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:26.821083 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18e30f-0b9c-4198-a907-54b0693510f8 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:26.821791 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18e30f-0b9c-4198-a907-54b0693510f8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:26.822264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18e30f-0b9c-4198-a907-54b0693510f8 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:26.824090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 148/293] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:26.829247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd58e02-1c5f-41c0-8b70-2788730fb0df None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:26.830071 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd58e02-1c5f-41c0-8b70-2788730fb0df None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:26.830660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd58e02-1c5f-41c0-8b70-2788730fb0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:26.841222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:26.863996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-fdd58e02-1c5f-41c0-8b70-2788730fb0df None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:26.865339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fdd58e02-1c5f-41c0-8b70-2788730fb0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:26.881863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f835a-0391-4051-a83e-a6136fdcd4da None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:26.923632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427707c-c04f-4df2-abe9-7588fc5ad416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:26.924537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427707c-c04f-4df2-abe9-7588fc5ad416 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:26.924825 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427707c-c04f-4df2-abe9-7588fc5ad416 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:26.925106 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427707c-c04f-4df2-abe9-7588fc5ad416 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:26.930385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427707c-c04f-4df2-abe9-7588fc5ad416 None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:26.932303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427707c-c04f-4df2-abe9-7588fc5ad416 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:26.934675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 146/294] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:26 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:26.941660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427707c-c04f-4df2-abe9-7588fc5ad416 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:27.014998 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 149/295] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:27.020810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da930a-e151-4c75-9ea9-5a0dab051153 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.021099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da930a-e151-4c75-9ea9-5a0dab051153 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.021381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da930a-e151-4c75-9ea9-5a0dab051153 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.023120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 150/296] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:27.035057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4414b-e4d2-4b6f-a7bc-e0e3a4972f0d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.035487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4414b-e4d2-4b6f-a7bc-e0e3a4972f0d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.035869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4414b-e4d2-4b6f-a7bc-e0e3a4972f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.059500 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a0e4414b-e4d2-4b6f-a7bc-e0e3a4972f0d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:27.061122 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a0e4414b-e4d2-4b6f-a7bc-e0e3a4972f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:27.083388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:27.083755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427707c-c04f-4df2-abe9-7588fc5ad416 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:27.135266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc77cfc-c639-40bb-ae87-80271c7e25a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:27.136004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc77cfc-c639-40bb-ae87-80271c7e25a3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.136285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc77cfc-c639-40bb-ae87-80271c7e25a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.136684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc77cfc-c639-40bb-ae87-80271c7e25a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.137529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc77cfc-c639-40bb-ae87-80271c7e25a3 None admin] RBAC: Adding query filter params (role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d, scope.project.id=62e3ae8270a64809a9778530c2b3b7f0, user.id=53aba92d86724ccf876f3f10db8ada11) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:27.137957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc77cfc-c639-40bb-ae87-80271c7e25a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:27.139917 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc77cfc-c639-40bb-ae87-80271c7e25a3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:27.142348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:27.148298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 147/297] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:26 2022] GET /identity/v3/role_assignments?user.id=53aba92d86724ccf876f3f10db8ada11&scope.project.id=62e3ae8270a64809a9778530c2b3b7f0&role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:27.148924 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc77cfc-c639-40bb-ae87-80271c7e25a3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:27.154267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 151/298] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:27.184426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4414b-e4d2-4b6f-a7bc-e0e3a4972f0d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:27.322683 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc77cfc-c639-40bb-ae87-80271c7e25a3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:27.349466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4414b-e4d2-4b6f-a7bc-e0e3a4972f0d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:27.364072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44491a08-1517-4f4b-b7e6-d67fd918f876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:27.365157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44491a08-1517-4f4b-b7e6-d67fd918f876 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.365440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44491a08-1517-4f4b-b7e6-d67fd918f876 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.365758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44491a08-1517-4f4b-b7e6-d67fd918f876 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.379265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44491a08-1517-4f4b-b7e6-d67fd918f876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-44491a08-1517-4f4b-b7e6-d67fd918f876 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:27.387436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:27.388300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:27.389066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:27.389066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:27.389066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:27.389066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:27.389066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:27.389066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:27.389066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:27.389066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:27.389066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:27.389066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:27.389066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:27.389066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:27.389066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 148/299] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:27.390365 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056575be-5fcc-4659-bb5d-3773c2432a6f None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.390365 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056575be-5fcc-4659-bb5d-3773c2432a6f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.390573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056575be-5fcc-4659-bb5d-3773c2432a6f None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.393668 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 149/300] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:27.394612 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056575be-5fcc-4659-bb5d-3773c2432a6f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:27.404070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd964cc8-ad61-4369-9c5c-6990b40e570e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:27.404879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd964cc8-ad61-4369-9c5c-6990b40e570e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.405167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd964cc8-ad61-4369-9c5c-6990b40e570e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.405444 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd964cc8-ad61-4369-9c5c-6990b40e570e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.407234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd964cc8-ad61-4369-9c5c-6990b40e570e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:27.409060 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd964cc8-ad61-4369-9c5c-6990b40e570e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:27.411770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 152/301] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:27.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8ab3f-8c9b-4e72-9c4a-9efc9881bd13 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.413502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8ab3f-8c9b-4e72-9c4a-9efc9881bd13 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.413807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8ab3f-8c9b-4e72-9c4a-9efc9881bd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.437489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a0e8ab3f-8c9b-4e72-9c4a-9efc9881bd13 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:27.439210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a0e8ab3f-8c9b-4e72-9c4a-9efc9881bd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:27.462641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:27.472447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:27.523508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 153/302] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:27.524688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e8ab3f-8c9b-4e72-9c4a-9efc9881bd13 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:27.534408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056575be-5fcc-4659-bb5d-3773c2432a6f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:27.572122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c3ed6-ecfa-435c-bd78-acfc9f944bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:27.572796 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c3ed6-ecfa-435c-bd78-acfc9f944bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.573078 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c3ed6-ecfa-435c-bd78-acfc9f944bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.573355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c3ed6-ecfa-435c-bd78-acfc9f944bf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.574124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c3ed6-ecfa-435c-bd78-acfc9f944bf2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:27.574428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c3ed6-ecfa-435c-bd78-acfc9f944bf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:27.575832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c3ed6-ecfa-435c-bd78-acfc9f944bf2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:27.584401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 150/303] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:27.585204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c3ed6-ecfa-435c-bd78-acfc9f944bf2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:27.639233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e8ab3f-8c9b-4e72-9c4a-9efc9881bd13 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:27.671981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95683b7-5e84-4191-984f-ee60f5753994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:27.672708 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95683b7-5e84-4191-984f-ee60f5753994 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.673168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95683b7-5e84-4191-984f-ee60f5753994 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.673447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95683b7-5e84-4191-984f-ee60f5753994 None admin] PATH_INFO: `/v3/groups` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.674170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95683b7-5e84-4191-984f-ee60f5753994 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:27.675635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95683b7-5e84-4191-984f-ee60f5753994 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:27.687030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c3ed6-ecfa-435c-bd78-acfc9f944bf2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:27.692290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 154/304] 10.222.0.79 () {60 vars in 1134 bytes} [Mon Jul 4 09:55:27 2022] POST /identity/v3/groups => generated 229 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:55:27.693096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95683b7-5e84-4191-984f-ee60f5753994 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:27.709404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:27.721834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:27.740028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8148bc2-03a8-41e2-9d47-92dc4e098a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:27.740811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8148bc2-03a8-41e2-9d47-92dc4e098a27 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.741101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8148bc2-03a8-41e2-9d47-92dc4e098a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.741379 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8148bc2-03a8-41e2-9d47-92dc4e098a27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.754072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8148bc2-03a8-41e2-9d47-92dc4e098a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-f8148bc2-03a8-41e2-9d47-92dc4e098a27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:27.762045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:27.763195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:27.764265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:27.764265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:27.764265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:27.764265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:27.764265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:27.764265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:27.764265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:27.764265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:27.764265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:27.764265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:27.764265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:27.764265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:27.764265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 151/305] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:27.765094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4032d-d4c6-471f-b60d-0b126bc55e62 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.765094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4032d-d4c6-471f-b60d-0b126bc55e62 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.765245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4032d-d4c6-471f-b60d-0b126bc55e62 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.766531 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 152/306] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:27.771662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4032d-d4c6-471f-b60d-0b126bc55e62 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:27.825124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95683b7-5e84-4191-984f-ee60f5753994 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:27.860411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb217b-7d77-453f-a3a6-5290ac7dd1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:27.861150 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb217b-7d77-453f-a3a6-5290ac7dd1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.861433 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb217b-7d77-453f-a3a6-5290ac7dd1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.861739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb217b-7d77-453f-a3a6-5290ac7dd1f3 None admin] PATH_INFO: `/v3/users/2cd0852169844208aaabb4f9e78398f1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.876773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb217b-7d77-453f-a3a6-5290ac7dd1f3 None admin] RBAC: Authorizing `identity:get_user(user_id=2cd0852169844208aaabb4f9e78398f1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:27.878587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb217b-7d77-453f-a3a6-5290ac7dd1f3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:27.880445 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 155/307] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:27 2022] GET /identity/v3/users/2cd0852169844208aaabb4f9e78398f1 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:27.885664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a8889-34c5-486d-8b90-9729620630e5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.885664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a8889-34c5-486d-8b90-9729620630e5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.885664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a8889-34c5-486d-8b90-9729620630e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.894398 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4032d-d4c6-471f-b60d-0b126bc55e62 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:27.899079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-810a8889-34c5-486d-8b90-9729620630e5 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:27.900297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-810a8889-34c5-486d-8b90-9729620630e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:27.931598 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8723c-a1af-4dcc-a3a4-7a755b4e167b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:27.932418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8723c-a1af-4dcc-a3a4-7a755b4e167b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:27.932704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8723c-a1af-4dcc-a3a4-7a755b4e167b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:27.932977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8723c-a1af-4dcc-a3a4-7a755b4e167b None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:27.934101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8723c-a1af-4dcc-a3a4-7a755b4e167b None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:27.934291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8723c-a1af-4dcc-a3a4-7a755b4e167b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:27.935693 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8723c-a1af-4dcc-a3a4-7a755b4e167b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:27.944849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 153/308] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:27.945570 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8723c-a1af-4dcc-a3a4-7a755b4e167b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:27.968091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:27.981895 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 156/309] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:27.982701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a8889-34c5-486d-8b90-9729620630e5 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:27.991595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:28.057152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8723c-a1af-4dcc-a3a4-7a755b4e167b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:28.091793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67f8ca-be03-4c2e-820b-0e7f07535bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:28.092696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67f8ca-be03-4c2e-820b-0e7f07535bdb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:28.092974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67f8ca-be03-4c2e-820b-0e7f07535bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:28.093248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67f8ca-be03-4c2e-820b-0e7f07535bdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:28.094094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67f8ca-be03-4c2e-820b-0e7f07535bdb None admin] RBAC: Adding query filter params (role.id=f420d4b9bfb64b0f8cd462e9d72bd24d, user.id=2cd0852169844208aaabb4f9e78398f1, scope.system=all) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:28.094398 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67f8ca-be03-4c2e-820b-0e7f07535bdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:28.095668 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a8889-34c5-486d-8b90-9729620630e5 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:28.096113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67f8ca-be03-4c2e-820b-0e7f07535bdb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:28.101838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 154/310] 10.222.0.79 () {56 vars in 1308 bytes} [Mon Jul 4 09:55:27 2022] GET /identity/v3/role_assignments?user.id=2cd0852169844208aaabb4f9e78398f1&scope.system=all&role.id=f420d4b9bfb64b0f8cd462e9d72bd24d => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:28.102527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67f8ca-be03-4c2e-820b-0e7f07535bdb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:28.154113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f40a8a-93a9-4b29-9a03-eaa1b6d977f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:28.156167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f40a8a-93a9-4b29-9a03-eaa1b6d977f2 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:28.156687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f40a8a-93a9-4b29-9a03-eaa1b6d977f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:28.157686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f40a8a-93a9-4b29-9a03-eaa1b6d977f2 None admin] PATH_INFO: `/v3/users/06a5910838584896aa26ebcb0ff40d28` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:28.159743 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f40a8a-93a9-4b29-9a03-eaa1b6d977f2 None admin] RBAC: Authorizing `identity:get_user(user_id=06a5910838584896aa26ebcb0ff40d28)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:28.162013 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f40a8a-93a9-4b29-9a03-eaa1b6d977f2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:28.168690 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 157/311] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:27 2022] GET /identity/v3/users/06a5910838584896aa26ebcb0ff40d28 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:28.170453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f40a8a-93a9-4b29-9a03-eaa1b6d977f2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:28.221938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67f8ca-be03-4c2e-820b-0e7f07535bdb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:28.231791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:28.254329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:28.278955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454687c0-5fbb-4f82-8d23-5036214cb826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:28.280109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454687c0-5fbb-4f82-8d23-5036214cb826 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:28.280462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454687c0-5fbb-4f82-8d23-5036214cb826 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:28.280873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454687c0-5fbb-4f82-8d23-5036214cb826 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:28.292080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454687c0-5fbb-4f82-8d23-5036214cb826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:28.294404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f40a8a-93a9-4b29-9a03-eaa1b6d977f2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-454687c0-5fbb-4f82-8d23-5036214cb826 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:28.300369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:28.301438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:28.302503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:28.302503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:28.302503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:28.302503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:28.302503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:28.302503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:28.302503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:28.302503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:28.302503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:28.302503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:28.302503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:28.302503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:28.305648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 155/312] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:28.311011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454687c0-5fbb-4f82-8d23-5036214cb826 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:28.336438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c125c2-43d3-4f29-9c00-a0d1691c6ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:28.337399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c125c2-43d3-4f29-9c00-a0d1691c6ec2 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:28.337785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c125c2-43d3-4f29-9c00-a0d1691c6ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:28.338233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c125c2-43d3-4f29-9c00-a0d1691c6ec2 None admin] PATH_INFO: `/v3/system/users/06a5910838584896aa26ebcb0ff40d28/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:28.340043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c125c2-43d3-4f29-9c00-a0d1691c6ec2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=06a5910838584896aa26ebcb0ff40d28, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:28.341603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c125c2-43d3-4f29-9c00-a0d1691c6ec2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:28.354260 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 158/313] 10.222.0.79 () {58 vars in 1334 bytes} [Mon Jul 4 09:55:28 2022] PUT /identity/v3/system/users/06a5910838584896aa26ebcb0ff40d28/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:55:28.424132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454687c0-5fbb-4f82-8d23-5036214cb826 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:28.468986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386bd00-fd62-47aa-8892-fffd13e2def4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:28.470191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386bd00-fd62-47aa-8892-fffd13e2def4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:28.470656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386bd00-fd62-47aa-8892-fffd13e2def4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:28.471179 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386bd00-fd62-47aa-8892-fffd13e2def4 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:28.472123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386bd00-fd62-47aa-8892-fffd13e2def4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:28.472606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386bd00-fd62-47aa-8892-fffd13e2def4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:28.474297 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386bd00-fd62-47aa-8892-fffd13e2def4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:28.485024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 156/314] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:28.493742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c125c2-43d3-4f29-9c00-a0d1691c6ec2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:28.601420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c125c2-43d3-4f29-9c00-a0d1691c6ec2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:28.631197 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:28.636224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148c200-25d5-4fbb-a8de-88f6316d7cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:28.637117 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148c200-25d5-4fbb-a8de-88f6316d7cce None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:28.637463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148c200-25d5-4fbb-a8de-88f6316d7cce None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:28.637695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148c200-25d5-4fbb-a8de-88f6316d7cce None admin] PATH_INFO: `/v3/users/341b5c5a8b3e4662b9a08e0617382827` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:28.638987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148c200-25d5-4fbb-a8de-88f6316d7cce None admin] RBAC: Authorizing `identity:get_user(user_id=341b5c5a8b3e4662b9a08e0617382827)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:28.640547 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148c200-25d5-4fbb-a8de-88f6316d7cce None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:28.642913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 159/315] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:28 2022] GET /identity/v3/users/341b5c5a8b3e4662b9a08e0617382827 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:28.648171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386bd00-fd62-47aa-8892-fffd13e2def4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:28.678810 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:28.753055 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386bd00-fd62-47aa-8892-fffd13e2def4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:28.787566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5485668-b34d-4b7a-8ef7-43137ca25c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:28.788228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5485668-b34d-4b7a-8ef7-43137ca25c6a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:28.788514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5485668-b34d-4b7a-8ef7-43137ca25c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:28.788793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5485668-b34d-4b7a-8ef7-43137ca25c6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:28.861047 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5485668-b34d-4b7a-8ef7-43137ca25c6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:28.862579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5485668-b34d-4b7a-8ef7-43137ca25c6a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:28.890909 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:28.965641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 157/316] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:55:28.970809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148c200-25d5-4fbb-a8de-88f6316d7cce None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:29.074177 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148c200-25d5-4fbb-a8de-88f6316d7cce None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:29.109557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009cf5c-185b-4cf6-85fd-22c316dae893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:29.110611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009cf5c-185b-4cf6-85fd-22c316dae893 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:29.110898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009cf5c-185b-4cf6-85fd-22c316dae893 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:29.111171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009cf5c-185b-4cf6-85fd-22c316dae893 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:29.112364 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009cf5c-185b-4cf6-85fd-22c316dae893 None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:29.113835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009cf5c-185b-4cf6-85fd-22c316dae893 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:29.115842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 160/317] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:28 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:29.121705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5485668-b34d-4b7a-8ef7-43137ca25c6a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:29.222936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5485668-b34d-4b7a-8ef7-43137ca25c6a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:29.249581 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:29.257688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cabba-ed64-4cbb-8f8a-8fefba50f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:29.258480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cabba-ed64-4cbb-8f8a-8fefba50f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:29.258908 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cabba-ed64-4cbb-8f8a-8fefba50f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:29.259298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cabba-ed64-4cbb-8f8a-8fefba50f2c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:29.260324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261cabba-ed64-4cbb-8f8a-8fefba50f2c1 None admin] RBAC: Adding query filter params (role.id=f420d4b9bfb64b0f8cd462e9d72bd24d, scope.project.id=62e3ae8270a64809a9778530c2b3b7f0, user.id=341b5c5a8b3e4662b9a08e0617382827) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:29.260700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261cabba-ed64-4cbb-8f8a-8fefba50f2c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:29.262259 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261cabba-ed64-4cbb-8f8a-8fefba50f2c1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:29.268710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 158/318] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:29 2022] GET /identity/v3/role_assignments?user.id=341b5c5a8b3e4662b9a08e0617382827&scope.project.id=62e3ae8270a64809a9778530c2b3b7f0&role.id=f420d4b9bfb64b0f8cd462e9d72bd24d => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:29.783820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980197cc-13c2-4851-a354-d91141f204b6 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:29.784225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980197cc-13c2-4851-a354-d91141f204b6 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:29.784321 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980197cc-13c2-4851-a354-d91141f204b6 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:29.785979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 161/319] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:29.791201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cbb7c-f22b-4790-adf9-e9f666547386 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:29.791490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cbb7c-f22b-4790-adf9-e9f666547386 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:29.791765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cbb7c-f22b-4790-adf9-e9f666547386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:29.808861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-997cbb7c-f22b-4790-adf9-e9f666547386 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:29.810185 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-997cbb7c-f22b-4790-adf9-e9f666547386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:29.881730 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 159/320] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:29.914938 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980197cc-13c2-4851-a354-d91141f204b6 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:29.995238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:30.031367 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980197cc-13c2-4851-a354-d91141f204b6 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:30.067226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075ba6f-b857-49d4-87e1-1f8c1d863979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:30.067944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075ba6f-b857-49d4-87e1-1f8c1d863979 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:30.068200 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075ba6f-b857-49d4-87e1-1f8c1d863979 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:30.068493 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075ba6f-b857-49d4-87e1-1f8c1d863979 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:30.069719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075ba6f-b857-49d4-87e1-1f8c1d863979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:30.071224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075ba6f-b857-49d4-87e1-1f8c1d863979 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:30.077297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 162/321] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:30.078589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cbb7c-f22b-4790-adf9-e9f666547386 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:30.189212 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cbb7c-f22b-4790-adf9-e9f666547386 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:30.212646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:30.226250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef4f82-0f50-4fac-8427-7422c9a59d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:30.226933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef4f82-0f50-4fac-8427-7422c9a59d50 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:30.227213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef4f82-0f50-4fac-8427-7422c9a59d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:30.227486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef4f82-0f50-4fac-8427-7422c9a59d50 None admin] PATH_INFO: `/v3/groups` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:30.228172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef4f82-0f50-4fac-8427-7422c9a59d50 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:30.229648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef4f82-0f50-4fac-8427-7422c9a59d50 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:30.253656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 160/322] 10.222.0.79 () {60 vars in 1134 bytes} [Mon Jul 4 09:55:30 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:55:30.253887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb4315-3967-46c2-a49d-d7fba642b6ed None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:30.253887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb4315-3967-46c2-a49d-d7fba642b6ed None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:30.253887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb4315-3967-46c2-a49d-d7fba642b6ed None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:30.253887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 163/323] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:30.257032 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70010357-6fbf-46b4-bab0-f14f89381cc9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:30.257325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70010357-6fbf-46b4-bab0-f14f89381cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:30.257645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70010357-6fbf-46b4-bab0-f14f89381cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:30.275427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-70010357-6fbf-46b4-bab0-f14f89381cc9 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:30.276649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-70010357-6fbf-46b4-bab0-f14f89381cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:30.349754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 161/324] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:30.372538 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb4315-3967-46c2-a49d-d7fba642b6ed None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:30.471606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d2845-94a7-46de-bf82-2965e04585be None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:30.471889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d2845-94a7-46de-bf82-2965e04585be None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:30.472169 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d2845-94a7-46de-bf82-2965e04585be None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:30.474589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 162/325] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:30.479930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca75e2-9039-468b-8507-53a9e3d18882 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:30.480282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca75e2-9039-468b-8507-53a9e3d18882 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:30.480557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca75e2-9039-468b-8507-53a9e3d18882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:30.483248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb4315-3967-46c2-a49d-d7fba642b6ed None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:30.500175 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-feca75e2-9039-468b-8507-53a9e3d18882 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:30.501452 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-feca75e2-9039-468b-8507-53a9e3d18882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:30.529030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e6e15-2191-4dba-87dc-3e1aa6365fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:30.530830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e6e15-2191-4dba-87dc-3e1aa6365fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:30.530830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e6e15-2191-4dba-87dc-3e1aa6365fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:30.530830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e6e15-2191-4dba-87dc-3e1aa6365fe5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:30.536947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:30.545207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e6e15-2191-4dba-87dc-3e1aa6365fe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-818e6e15-2191-4dba-87dc-3e1aa6365fe5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:30.556002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:30.557105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:30.558102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:30.558102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:30.558102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:30.558102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:30.558102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:30.558102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:30.558102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:30.558102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:30.558102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:30.558102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:30.558102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:30.558102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:30.560831 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 164/326] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:30.567246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e6e15-2191-4dba-87dc-3e1aa6365fe5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:30.588173 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 163/327] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:30.612576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca75e2-9039-468b-8507-53a9e3d18882 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:30.668447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:30.691869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e6e15-2191-4dba-87dc-3e1aa6365fe5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:30.733666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056617e-73de-4822-b858-9c1201f530f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:30.734599 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056617e-73de-4822-b858-9c1201f530f2 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:30.735100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056617e-73de-4822-b858-9c1201f530f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:30.735471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056617e-73de-4822-b858-9c1201f530f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:30.736320 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056617e-73de-4822-b858-9c1201f530f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:30.736721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056617e-73de-4822-b858-9c1201f530f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:30.738504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056617e-73de-4822-b858-9c1201f530f2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:30.741548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca75e2-9039-468b-8507-53a9e3d18882 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:30.748586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 165/328] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:30.757510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056617e-73de-4822-b858-9c1201f530f2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:30.795718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9df9d-876b-4f94-8a47-cd6ef8185006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:30.796925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9df9d-876b-4f94-8a47-cd6ef8185006 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:30.797406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9df9d-876b-4f94-8a47-cd6ef8185006 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:30.798078 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9df9d-876b-4f94-8a47-cd6ef8185006 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:30.814101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9df9d-876b-4f94-8a47-cd6ef8185006 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-90a9df9d-876b-4f94-8a47-cd6ef8185006 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:30.824838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:30.825789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:30.826576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:30.826576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:30.826576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:30.826576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:30.826576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:30.826576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:30.826576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:30.826576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:30.826576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:30.826576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:30.826576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:30.826576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:30.828094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 164/329] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:30.834449 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9df9d-876b-4f94-8a47-cd6ef8185006 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:30.839868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:30.889188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056617e-73de-4822-b858-9c1201f530f2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:30.927998 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defd7e7-acaa-46d2-8320-628033e4be10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:30.928931 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defd7e7-acaa-46d2-8320-628033e4be10 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:30.929214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defd7e7-acaa-46d2-8320-628033e4be10 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:30.929481 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defd7e7-acaa-46d2-8320-628033e4be10 None admin] PATH_INFO: `/v3/users/2cd0852169844208aaabb4f9e78398f1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:30.930897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defd7e7-acaa-46d2-8320-628033e4be10 None admin] RBAC: Authorizing `identity:get_user(user_id=2cd0852169844208aaabb4f9e78398f1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:30.932434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defd7e7-acaa-46d2-8320-628033e4be10 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:30.934439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 166/330] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:30 2022] GET /identity/v3/users/2cd0852169844208aaabb4f9e78398f1 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:30.946111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defd7e7-acaa-46d2-8320-628033e4be10 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:30.983496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9df9d-876b-4f94-8a47-cd6ef8185006 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:30.985602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:31.022455 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82f4d9-fe20-49dd-a344-852a22ce6e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:31.023568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82f4d9-fe20-49dd-a344-852a22ce6e63 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:31.024056 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82f4d9-fe20-49dd-a344-852a22ce6e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:31.024492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82f4d9-fe20-49dd-a344-852a22ce6e63 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:31.025437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82f4d9-fe20-49dd-a344-852a22ce6e63 None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:31.025937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82f4d9-fe20-49dd-a344-852a22ce6e63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:31.030692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82f4d9-fe20-49dd-a344-852a22ce6e63 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:31.044251 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 165/331] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:31.050387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82f4d9-fe20-49dd-a344-852a22ce6e63 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:31.062917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defd7e7-acaa-46d2-8320-628033e4be10 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:31.091531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:31.103457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c7a67-753e-4a02-b676-7b5c9753d033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:31.104615 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c7a67-753e-4a02-b676-7b5c9753d033 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:31.105271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c7a67-753e-4a02-b676-7b5c9753d033 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:31.105640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c7a67-753e-4a02-b676-7b5c9753d033 None admin] PATH_INFO: `/v3/system/users/2cd0852169844208aaabb4f9e78398f1/roles/f420d4b9bfb64b0f8cd462e9d72bd24d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:31.107507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c7a67-753e-4a02-b676-7b5c9753d033 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2cd0852169844208aaabb4f9e78398f1, role_id=f420d4b9bfb64b0f8cd462e9d72bd24d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:31.109112 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c7a67-753e-4a02-b676-7b5c9753d033 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:31.128822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 167/332] 10.222.0.79 () {58 vars in 1334 bytes} [Mon Jul 4 09:55:30 2022] PUT /identity/v3/system/users/2cd0852169844208aaabb4f9e78398f1/roles/f420d4b9bfb64b0f8cd462e9d72bd24d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:55:31.176486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82f4d9-fe20-49dd-a344-852a22ce6e63 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:31.217129 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a2d82-81bd-4a26-947d-65f4c5e0a849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:31.218064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a2d82-81bd-4a26-947d-65f4c5e0a849 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:31.218350 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a2d82-81bd-4a26-947d-65f4c5e0a849 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:31.218635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a2d82-81bd-4a26-947d-65f4c5e0a849 None admin] PATH_INFO: `/v3/users/06a5910838584896aa26ebcb0ff40d28` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:31.220456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a2d82-81bd-4a26-947d-65f4c5e0a849 None admin] RBAC: Authorizing `identity:get_user(user_id=06a5910838584896aa26ebcb0ff40d28)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:31.222255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a2d82-81bd-4a26-947d-65f4c5e0a849 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:31.224574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 166/333] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:31 2022] GET /identity/v3/users/06a5910838584896aa26ebcb0ff40d28 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:31.230838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c7a67-753e-4a02-b676-7b5c9753d033 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:31.368013 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c7a67-753e-4a02-b676-7b5c9753d033 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:31.405146 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341c3ea-0a86-4e7e-8280-00cb43e42f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:31.405844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341c3ea-0a86-4e7e-8280-00cb43e42f9d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:31.406131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341c3ea-0a86-4e7e-8280-00cb43e42f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:31.406406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341c3ea-0a86-4e7e-8280-00cb43e42f9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:31.407642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341c3ea-0a86-4e7e-8280-00cb43e42f9d None admin] RBAC: Adding query filter params (role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d, user.id=06a5910838584896aa26ebcb0ff40d28, scope.system=all) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:31.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341c3ea-0a86-4e7e-8280-00cb43e42f9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:31.410929 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341c3ea-0a86-4e7e-8280-00cb43e42f9d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:31.425501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 168/334] 10.222.0.79 () {56 vars in 1308 bytes} [Mon Jul 4 09:55:31 2022] GET /identity/v3/role_assignments?user.id=06a5910838584896aa26ebcb0ff40d28&scope.system=all&role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:32.779767 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc54be-2161-4c79-a170-2d66bcf7bda3 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:32.780291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc54be-2161-4c79-a170-2d66bcf7bda3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:32.780367 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc54be-2161-4c79-a170-2d66bcf7bda3 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:32.782301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 167/335] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:32.787576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5d9c1-0b72-4076-a892-4bbd177d4dbd None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:32.787988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5d9c1-0b72-4076-a892-4bbd177d4dbd None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:32.788156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5d9c1-0b72-4076-a892-4bbd177d4dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:32.808637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7dc5d9c1-0b72-4076-a892-4bbd177d4dbd None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:32.809868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7dc5d9c1-0b72-4076-a892-4bbd177d4dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:32.885836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:32.911191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 169/336] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:32.949405 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc54be-2161-4c79-a170-2d66bcf7bda3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:32.998803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:33.084149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc54be-2161-4c79-a170-2d66bcf7bda3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:33.120780 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fdeaf-4939-4044-af6d-edfe8466dd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:33.127280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fdeaf-4939-4044-af6d-edfe8466dd6a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:33.127823 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fdeaf-4939-4044-af6d-edfe8466dd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:33.128294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fdeaf-4939-4044-af6d-edfe8466dd6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:33.151102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fdeaf-4939-4044-af6d-edfe8466dd6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-108fdeaf-4939-4044-af6d-edfe8466dd6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:33.159493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:33.160349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:33.161134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:33.161134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:33.161134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:33.161134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:33.161134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:33.161134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:33.161134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:33.161134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:33.161134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:33.161134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:33.161134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:33.161134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:33.163531 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 168/337] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:33.169589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5d9c1-0b72-4076-a892-4bbd177d4dbd None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:33.221829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775ee8c-20c6-4e7d-bc9d-488621268801 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:33.222425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775ee8c-20c6-4e7d-bc9d-488621268801 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:33.222873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775ee8c-20c6-4e7d-bc9d-488621268801 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:33.227349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 169/338] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:33.232963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412eb41a-cf24-435a-aa69-bd169293922c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:33.233505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412eb41a-cf24-435a-aa69-bd169293922c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:33.234038 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412eb41a-cf24-435a-aa69-bd169293922c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:33.266134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-412eb41a-cf24-435a-aa69-bd169293922c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:33.269175 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-412eb41a-cf24-435a-aa69-bd169293922c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:33.352027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5d9c1-0b72-4076-a892-4bbd177d4dbd None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:33.401000 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b6c0c-b6e5-4190-b701-5b716011ccf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:33.402110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b6c0c-b6e5-4190-b701-5b716011ccf6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:33.402596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b6c0c-b6e5-4190-b701-5b716011ccf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:33.403072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b6c0c-b6e5-4190-b701-5b716011ccf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:33.404027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b6c0c-b6e5-4190-b701-5b716011ccf6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:33.404548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b6c0c-b6e5-4190-b701-5b716011ccf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:33.408605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b6c0c-b6e5-4190-b701-5b716011ccf6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:33.418407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 170/339] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:33.424575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b6c0c-b6e5-4190-b701-5b716011ccf6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:33.424723 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 170/340] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:33.463161 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412eb41a-cf24-435a-aa69-bd169293922c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:33.473654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:33.506781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:33.544200 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b6c0c-b6e5-4190-b701-5b716011ccf6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:33.586001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3ed20-131d-430c-a419-1352f0e54553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:33.586934 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3ed20-131d-430c-a419-1352f0e54553 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:33.587388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3ed20-131d-430c-a419-1352f0e54553 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:33.587810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3ed20-131d-430c-a419-1352f0e54553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:33.663034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3ed20-131d-430c-a419-1352f0e54553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:33.664844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3ed20-131d-430c-a419-1352f0e54553 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:33.707854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412eb41a-cf24-435a-aa69-bd169293922c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:33.708400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:33.773275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 171/341] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:55:33.779961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3ed20-131d-430c-a419-1352f0e54553 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:33.805004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d734f8-8f71-4703-bca3-ad4d0d928b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:33.806569 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d734f8-8f71-4703-bca3-ad4d0d928b40 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:33.807180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d734f8-8f71-4703-bca3-ad4d0d928b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:33.810007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d734f8-8f71-4703-bca3-ad4d0d928b40 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:33.827917 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d734f8-8f71-4703-bca3-ad4d0d928b40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-04d734f8-8f71-4703-bca3-ad4d0d928b40 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:33.845875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:33.846861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:33.847645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:33.847645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:33.847645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:33.847645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:33.847645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:33.847645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:33.847645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:33.847645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:33.847645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:33.847645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:33.847645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:33.847645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:33.850114 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 171/342] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:55:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:33.856461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d734f8-8f71-4703-bca3-ad4d0d928b40 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:33.891605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3ed20-131d-430c-a419-1352f0e54553 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:33.931322 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691ad6c-7d98-4dc4-a8dd-3caba3d236df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:33.932134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691ad6c-7d98-4dc4-a8dd-3caba3d236df None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:33.932419 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691ad6c-7d98-4dc4-a8dd-3caba3d236df None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:33.932696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691ad6c-7d98-4dc4-a8dd-3caba3d236df None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:33.934349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691ad6c-7d98-4dc4-a8dd-3caba3d236df None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:33.936153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691ad6c-7d98-4dc4-a8dd-3caba3d236df None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:33.938788 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 172/343] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:33 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:33.944959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691ad6c-7d98-4dc4-a8dd-3caba3d236df None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:33.963716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:34.060635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691ad6c-7d98-4dc4-a8dd-3caba3d236df None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:34.061068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:34.100342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e36590-9fa8-4006-86de-6cea320fd85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:34.101402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d734f8-8f71-4703-bca3-ad4d0d928b40 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:34.101603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e36590-9fa8-4006-86de-6cea320fd85f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:34.101871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e36590-9fa8-4006-86de-6cea320fd85f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:34.102148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e36590-9fa8-4006-86de-6cea320fd85f None admin] PATH_INFO: `/v3/groups/41e3de5fa88e4d14a42cbd73510dee66` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:34.109583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e36590-9fa8-4006-86de-6cea320fd85f None admin] RBAC: Authorizing `identity:get_group(group_id=41e3de5fa88e4d14a42cbd73510dee66)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:34.112159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e36590-9fa8-4006-86de-6cea320fd85f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:34.114303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 173/344] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:33 2022] GET /identity/v3/groups/41e3de5fa88e4d14a42cbd73510dee66 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:34.118919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd566c-6a62-4a0c-8534-f65d22ce6419 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:34.118919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd566c-6a62-4a0c-8534-f65d22ce6419 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:34.118919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd566c-6a62-4a0c-8534-f65d22ce6419 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:34.118919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 174/345] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:34.124538 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61b189-1247-4937-aace-48dd85204dc4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:34.125100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61b189-1247-4937-aace-48dd85204dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:34.127970 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61b189-1247-4937-aace-48dd85204dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:34.150303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ea61b189-1247-4937-aace-48dd85204dc4 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:34.151887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ea61b189-1247-4937-aace-48dd85204dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:34.170128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09e30f-7314-4268-9928-76c44daab5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:34.170977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09e30f-7314-4268-9928-76c44daab5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:34.171361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09e30f-7314-4268-9928-76c44daab5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:34.171736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09e30f-7314-4268-9928-76c44daab5f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:34.172861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09e30f-7314-4268-9928-76c44daab5f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:34.173294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09e30f-7314-4268-9928-76c44daab5f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:34.175144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09e30f-7314-4268-9928-76c44daab5f3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:34.195017 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 172/346] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:34.195995 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09e30f-7314-4268-9928-76c44daab5f3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:34.238056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 175/347] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:34.239227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61b189-1247-4937-aace-48dd85204dc4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:34.242314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:34.381365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61b189-1247-4937-aace-48dd85204dc4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:34.390653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09e30f-7314-4268-9928-76c44daab5f3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:34.425329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce44a8-29ca-4bdf-b314-bef12621526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:34.426291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce44a8-29ca-4bdf-b314-bef12621526f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:34.426962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce44a8-29ca-4bdf-b314-bef12621526f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:34.427387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce44a8-29ca-4bdf-b314-bef12621526f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:34.433837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b26290-e01f-4426-a92b-b5840c74c4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:34.434788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b26290-e01f-4426-a92b-b5840c74c4cc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:34.435285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b26290-e01f-4426-a92b-b5840c74c4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:34.435753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b26290-e01f-4426-a92b-b5840c74c4cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:34.436895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b26290-e01f-4426-a92b-b5840c74c4cc None admin] RBAC: Adding query filter params (group.id=41e3de5fa88e4d14a42cbd73510dee66, role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d, scope.project.id=27186cff1e94461099e5d6cc062b1abd) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:34.437404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b26290-e01f-4426-a92b-b5840c74c4cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:34.439132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b26290-e01f-4426-a92b-b5840c74c4cc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:34.453984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 173/348] 10.222.0.79 () {56 vars in 1376 bytes} [Mon Jul 4 09:55:34 2022] GET /identity/v3/role_assignments?group.id=41e3de5fa88e4d14a42cbd73510dee66&scope.project.id=27186cff1e94461099e5d6cc062b1abd&role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:34.455373 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b26290-e01f-4426-a92b-b5840c74c4cc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:34.472454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:34.513476 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce44a8-29ca-4bdf-b314-bef12621526f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:34.515889 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce44a8-29ca-4bdf-b314-bef12621526f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:34.522149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:34.646225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 176/349] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:34.648154 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ccfce-ddf2-4084-95fe-a2c1995d587f None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:34.648589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ccfce-ddf2-4084-95fe-a2c1995d587f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:34.649008 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ccfce-ddf2-4084-95fe-a2c1995d587f None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:34.651015 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 177/350] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:34.657656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b26290-e01f-4426-a92b-b5840c74c4cc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:34.669147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a2563-4c7e-46c1-b7d5-3d8f1ead54ab None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:34.669791 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a2563-4c7e-46c1-b7d5-3d8f1ead54ab None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:34.670285 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a2563-4c7e-46c1-b7d5-3d8f1ead54ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:34.702329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-685a2563-4c7e-46c1-b7d5-3d8f1ead54ab None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:34.705118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-685a2563-4c7e-46c1-b7d5-3d8f1ead54ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:34.731601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2483945-3e3d-4b2b-95d4-a9e3d9f7d706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:34.732677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2483945-3e3d-4b2b-95d4-a9e3d9f7d706 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:34.733202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2483945-3e3d-4b2b-95d4-a9e3d9f7d706 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:34.733667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2483945-3e3d-4b2b-95d4-a9e3d9f7d706 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:34.746757 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2483945-3e3d-4b2b-95d4-a9e3d9f7d706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b2483945-3e3d-4b2b-95d4-a9e3d9f7d706 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:34.755252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:34.756315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 174/351] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:34.760646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2483945-3e3d-4b2b-95d4-a9e3d9f7d706 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:34.789861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 178/352] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:34.790334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c8649-28ec-4560-8314-b085fbfcaf91 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:34.790623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c8649-28ec-4560-8314-b085fbfcaf91 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:34.790903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c8649-28ec-4560-8314-b085fbfcaf91 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:34.793653 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 179/353] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:34.793653 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c8649-28ec-4560-8314-b085fbfcaf91 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:34.845121 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:34.858070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:34.895966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2483945-3e3d-4b2b-95d4-a9e3d9f7d706 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:34.923151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c8649-28ec-4560-8314-b085fbfcaf91 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:34.933302 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb42391-57fc-4dbb-a40c-34d07c95a0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:34.934440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb42391-57fc-4dbb-a40c-34d07c95a0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:34.934865 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb42391-57fc-4dbb-a40c-34d07c95a0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:34.935282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb42391-57fc-4dbb-a40c-34d07c95a0f8 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:34.936652 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb42391-57fc-4dbb-a40c-34d07c95a0f8 None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:34.939892 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb42391-57fc-4dbb-a40c-34d07c95a0f8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:34.941881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 175/354] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:34 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:34.943325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533f1fc-00cd-45ae-8901-0c60a9bd0801 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:34.943846 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533f1fc-00cd-45ae-8901-0c60a9bd0801 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:34.944338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533f1fc-00cd-45ae-8901-0c60a9bd0801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:34.961370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847a344-4142-43db-b0a8-9f9c982c55c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:34.962291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847a344-4142-43db-b0a8-9f9c982c55c5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:34.962715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847a344-4142-43db-b0a8-9f9c982c55c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:34.962873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d533f1fc-00cd-45ae-8901-0c60a9bd0801 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:34.963289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847a344-4142-43db-b0a8-9f9c982c55c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:34.964190 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847a344-4142-43db-b0a8-9f9c982c55c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:34.964300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d533f1fc-00cd-45ae-8901-0c60a9bd0801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:34.964738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847a344-4142-43db-b0a8-9f9c982c55c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:34.966312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847a344-4142-43db-b0a8-9f9c982c55c5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:34.976007 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 180/355] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:34.977105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847a344-4142-43db-b0a8-9f9c982c55c5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:35.040277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 176/356] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:35.041302 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533f1fc-00cd-45ae-8901-0c60a9bd0801 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:35.078227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:35.086620 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847a344-4142-43db-b0a8-9f9c982c55c5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:35.113853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:35.122513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2ee1f-bf22-45b4-8736-077f9956ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:35.123712 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2ee1f-bf22-45b4-8736-077f9956ce5f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:35.124192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2ee1f-bf22-45b4-8736-077f9956ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:35.124663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2ee1f-bf22-45b4-8736-077f9956ce5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:35.139401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2ee1f-bf22-45b4-8736-077f9956ce5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-05c2ee1f-bf22-45b4-8736-077f9956ce5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:35.148222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:35.149170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:35.149997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:35.149997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:35.149997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:35.149997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:35.149997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:35.149997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:35.149997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:35.149997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:35.149997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:35.149997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:35.149997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:55:35.149997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:35.155881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 181/357] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:35.155881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2ee1f-bf22-45b4-8736-077f9956ce5f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:35.163663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533f1fc-00cd-45ae-8901-0c60a9bd0801 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:35.200551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76593f-9ec2-4831-a93f-c95f4d8fab1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:35.201495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76593f-9ec2-4831-a93f-c95f4d8fab1e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:35.201811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76593f-9ec2-4831-a93f-c95f4d8fab1e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:35.202087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76593f-9ec2-4831-a93f-c95f4d8fab1e None admin] PATH_INFO: `/v3/groups/41e3de5fa88e4d14a42cbd73510dee66` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:35.203263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76593f-9ec2-4831-a93f-c95f4d8fab1e None admin] RBAC: Authorizing `identity:get_group(group_id=41e3de5fa88e4d14a42cbd73510dee66)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:35.205553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76593f-9ec2-4831-a93f-c95f4d8fab1e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:35.207524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 177/358] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:35 2022] GET /identity/v3/groups/41e3de5fa88e4d14a42cbd73510dee66 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:35.208493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76593f-9ec2-4831-a93f-c95f4d8fab1e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:35.269579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2ee1f-bf22-45b4-8736-077f9956ce5f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:35.307402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b6020-3255-4efa-80b8-423090ddef5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:35.308246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b6020-3255-4efa-80b8-423090ddef5b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:35.308443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b6020-3255-4efa-80b8-423090ddef5b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:35.308730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b6020-3255-4efa-80b8-423090ddef5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:35.319808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76593f-9ec2-4831-a93f-c95f4d8fab1e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:35.339113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:35.366195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c94759-969e-4ed9-91c0-24d626feb238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:35.368751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94759-969e-4ed9-91c0-24d626feb238 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:35.368751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94759-969e-4ed9-91c0-24d626feb238 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:35.368751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94759-969e-4ed9-91c0-24d626feb238 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:35.370771 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:35.381422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c94759-969e-4ed9-91c0-24d626feb238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-28c94759-969e-4ed9-91c0-24d626feb238 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:35.389567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:35.390591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:35.391497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:35.391497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:35.391497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:35.391497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:35.391497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:35.391497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:35.391497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:35.391497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:35.391497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:35.391497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:35.391497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:35.391497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:35.391497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 178/359] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:55:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:35.393882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c94759-969e-4ed9-91c0-24d626feb238 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:35.409964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b6020-3255-4efa-80b8-423090ddef5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:35.412162 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b6020-3255-4efa-80b8-423090ddef5b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:35.517633 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c94759-969e-4ed9-91c0-24d626feb238 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:35.533545 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 182/360] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:35.535059 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b6020-3255-4efa-80b8-423090ddef5b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:35.554914 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc9082-9289-49a8-8552-4b4045819106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:35.555594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc9082-9289-49a8-8552-4b4045819106 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:35.555879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc9082-9289-49a8-8552-4b4045819106 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:35.556156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc9082-9289-49a8-8552-4b4045819106 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:35.556920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc9082-9289-49a8-8552-4b4045819106 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:35.557207 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc9082-9289-49a8-8552-4b4045819106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:35.558740 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc9082-9289-49a8-8552-4b4045819106 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:35.568663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 179/361] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:35.569985 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8675b5f-286c-4808-85e7-10d9cc4e1567 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:35.570272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8675b5f-286c-4808-85e7-10d9cc4e1567 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:35.570555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8675b5f-286c-4808-85e7-10d9cc4e1567 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:35.573082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 180/362] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:35.573828 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8675b5f-286c-4808-85e7-10d9cc4e1567 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:35.611037 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:35.647178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:35.667647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b6020-3255-4efa-80b8-423090ddef5b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:35.708717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8675b5f-286c-4808-85e7-10d9cc4e1567 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:35.709860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50aad8d-0b55-4521-823f-c88ddcf6533f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:35.710733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50aad8d-0b55-4521-823f-c88ddcf6533f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:35.711223 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50aad8d-0b55-4521-823f-c88ddcf6533f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:35.711501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50aad8d-0b55-4521-823f-c88ddcf6533f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:35.712349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50aad8d-0b55-4521-823f-c88ddcf6533f None admin] RBAC: Adding query filter params (group.id=41e3de5fa88e4d14a42cbd73510dee66, role.id=78a15fc7951d4f28beb57f54dace457d, scope.project.id=27186cff1e94461099e5d6cc062b1abd) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:35.712669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50aad8d-0b55-4521-823f-c88ddcf6533f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:35.714185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50aad8d-0b55-4521-823f-c88ddcf6533f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:35.720538 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 183/363] 10.222.0.79 () {56 vars in 1376 bytes} [Mon Jul 4 09:55:35 2022] GET /identity/v3/role_assignments?group.id=41e3de5fa88e4d14a42cbd73510dee66&scope.project.id=27186cff1e94461099e5d6cc062b1abd&role.id=78a15fc7951d4f28beb57f54dace457d => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:35.721169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50aad8d-0b55-4521-823f-c88ddcf6533f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:35.747510 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e472e2-2c5b-4293-9bb9-7a1317152608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:35.749711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e472e2-2c5b-4293-9bb9-7a1317152608 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:35.750244 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e472e2-2c5b-4293-9bb9-7a1317152608 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:35.750795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e472e2-2c5b-4293-9bb9-7a1317152608 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:35.752070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e472e2-2c5b-4293-9bb9-7a1317152608 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:35.752589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e472e2-2c5b-4293-9bb9-7a1317152608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:35.754519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e472e2-2c5b-4293-9bb9-7a1317152608 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:35.766319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 181/364] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:35.767633 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e472e2-2c5b-4293-9bb9-7a1317152608 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:35.858245 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50aad8d-0b55-4521-823f-c88ddcf6533f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:35.883594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:35.900533 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e472e2-2c5b-4293-9bb9-7a1317152608 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:35.914293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfe5e6-5452-4a4b-869a-d9e42e085ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:35.916299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfe5e6-5452-4a4b-869a-d9e42e085ced None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:35.916787 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfe5e6-5452-4a4b-869a-d9e42e085ced None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:35.917258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfe5e6-5452-4a4b-869a-d9e42e085ced None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:35.919553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfe5e6-5452-4a4b-869a-d9e42e085ced None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:35.921947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfe5e6-5452-4a4b-869a-d9e42e085ced None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:35.924633 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 184/365] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:35 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:35.925400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:35.929548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e6f5f-01eb-477b-af10-3016879a12da None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:35.929890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e6f5f-01eb-477b-af10-3016879a12da None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:35.930174 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e6f5f-01eb-477b-af10-3016879a12da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:35.947804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a6723-14da-4ed2-991b-d752b4b47b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:35.948693 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a6723-14da-4ed2-991b-d752b4b47b31 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:35.948978 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a6723-14da-4ed2-991b-d752b4b47b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:35.949254 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a6723-14da-4ed2-991b-d752b4b47b31 None admin] PATH_INFO: `/v3/users/2cd0852169844208aaabb4f9e78398f1` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:35.950017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-969e6f5f-01eb-477b-af10-3016879a12da None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:35.950483 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a6723-14da-4ed2-991b-d752b4b47b31 None admin] RBAC: Authorizing `identity:get_user(user_id=2cd0852169844208aaabb4f9e78398f1)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:35.951266 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-969e6f5f-01eb-477b-af10-3016879a12da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:35.951957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a6723-14da-4ed2-991b-d752b4b47b31 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:35.954589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 182/366] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:55:35 2022] GET /identity/v3/users/2cd0852169844208aaabb4f9e78398f1 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:35.959638 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a6723-14da-4ed2-991b-d752b4b47b31 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:36.037668 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 185/367] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:36.038678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e6f5f-01eb-477b-af10-3016879a12da None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:36.104753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a6723-14da-4ed2-991b-d752b4b47b31 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:36.150316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b367a8-2299-4ab7-9fa6-9a20803ddb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:36.150316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b367a8-2299-4ab7-9fa6-9a20803ddb83 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:36.150316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b367a8-2299-4ab7-9fa6-9a20803ddb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:36.150316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b367a8-2299-4ab7-9fa6-9a20803ddb83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:36.150654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e6f5f-01eb-477b-af10-3016879a12da None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:36.170215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:36.186496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e2648-276c-4aed-97ba-dbefd569893f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:36.187378 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e2648-276c-4aed-97ba-dbefd569893f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:36.187660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e2648-276c-4aed-97ba-dbefd569893f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:36.187931 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e2648-276c-4aed-97ba-dbefd569893f None admin] PATH_INFO: `/v3/groups/41e3de5fa88e4d14a42cbd73510dee66` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:36.189170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e2648-276c-4aed-97ba-dbefd569893f None admin] RBAC: Authorizing `identity:get_group(group_id=41e3de5fa88e4d14a42cbd73510dee66)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:36.191172 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e2648-276c-4aed-97ba-dbefd569893f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:36.192852 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 186/368] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:36 2022] GET /identity/v3/groups/41e3de5fa88e4d14a42cbd73510dee66 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:36.194272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e2648-276c-4aed-97ba-dbefd569893f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:36.209251 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:36.224458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b367a8-2299-4ab7-9fa6-9a20803ddb83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:36.226527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b367a8-2299-4ab7-9fa6-9a20803ddb83 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:36.325271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e2648-276c-4aed-97ba-dbefd569893f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:36.365072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 183/369] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:36.366102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b367a8-2299-4ab7-9fa6-9a20803ddb83 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:36.371979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781ad5e-6c7a-433b-bf26-dfbbc5593dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:36.372646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781ad5e-6c7a-433b-bf26-dfbbc5593dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:36.372927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781ad5e-6c7a-433b-bf26-dfbbc5593dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:36.373204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781ad5e-6c7a-433b-bf26-dfbbc5593dd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:36.374064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781ad5e-6c7a-433b-bf26-dfbbc5593dd7 None admin] RBAC: Adding query filter params (role.id=f420d4b9bfb64b0f8cd462e9d72bd24d, user.id=2cd0852169844208aaabb4f9e78398f1, scope.system=all) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:36.374377 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781ad5e-6c7a-433b-bf26-dfbbc5593dd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:36.376456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781ad5e-6c7a-433b-bf26-dfbbc5593dd7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:36.393131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 187/370] 10.222.0.79 () {56 vars in 1308 bytes} [Mon Jul 4 09:55:36 2022] GET /identity/v3/role_assignments?user.id=2cd0852169844208aaabb4f9e78398f1&scope.system=all&role.id=f420d4b9bfb64b0f8cd462e9d72bd24d => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:36.394049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781ad5e-6c7a-433b-bf26-dfbbc5593dd7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:36.480963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:36.535751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:36.565369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b367a8-2299-4ab7-9fa6-9a20803ddb83 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:36.605824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00d560-c5e2-4271-aeaa-d442335bc01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:36.606375 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00d560-c5e2-4271-aeaa-d442335bc01b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:36.606654 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00d560-c5e2-4271-aeaa-d442335bc01b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:36.606926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00d560-c5e2-4271-aeaa-d442335bc01b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:36.609598 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781ad5e-6c7a-433b-bf26-dfbbc5593dd7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:36.618852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00d560-c5e2-4271-aeaa-d442335bc01b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1d00d560-c5e2-4271-aeaa-d442335bc01b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:36.626575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:36.627444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:36.628696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:36.628696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:36.628696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:36.628696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:36.628696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:36.628696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:36.628696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:36.628696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:36.628696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:36.628696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:36.628696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:36.628696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:36.628696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 184/371] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:55:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:36.629349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00d560-c5e2-4271-aeaa-d442335bc01b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:36.653072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a3c20-4938-4472-be3f-36f2c55c3c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:36.653786 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a3c20-4938-4472-be3f-36f2c55c3c45 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:36.654070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a3c20-4938-4472-be3f-36f2c55c3c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:36.654346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a3c20-4938-4472-be3f-36f2c55c3c45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:36.655200 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a3c20-4938-4472-be3f-36f2c55c3c45 None admin] RBAC: Adding query filter params (group.id=41e3de5fa88e4d14a42cbd73510dee66, role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d, scope.project.id=62e3ae8270a64809a9778530c2b3b7f0) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:36.655515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a3c20-4938-4472-be3f-36f2c55c3c45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:36.657120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a3c20-4938-4472-be3f-36f2c55c3c45 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:36.666141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 188/372] 10.222.0.79 () {56 vars in 1376 bytes} [Mon Jul 4 09:55:36 2022] GET /identity/v3/role_assignments?group.id=41e3de5fa88e4d14a42cbd73510dee66&scope.project.id=62e3ae8270a64809a9778530c2b3b7f0&role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 260 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:36.666974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a3c20-4938-4472-be3f-36f2c55c3c45 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:36.744061 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00d560-c5e2-4271-aeaa-d442335bc01b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:36.781303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b994b-4ebd-4e04-be61-c2fb20f4aef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:36.783415 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b994b-4ebd-4e04-be61-c2fb20f4aef2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:36.783799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b994b-4ebd-4e04-be61-c2fb20f4aef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:36.784178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b994b-4ebd-4e04-be61-c2fb20f4aef2 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:36.785653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b994b-4ebd-4e04-be61-c2fb20f4aef2 None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:36.787265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b994b-4ebd-4e04-be61-c2fb20f4aef2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:36.789124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 185/373] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:36 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:36.814961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b994b-4ebd-4e04-be61-c2fb20f4aef2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:36.831892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a3c20-4938-4472-be3f-36f2c55c3c45 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:36.845365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:36.857740 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:36.874568 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6f67f-b05d-4cb9-9b99-185d57c4d2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:36.875272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6f67f-b05d-4cb9-9b99-185d57c4d2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:36.875584 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6f67f-b05d-4cb9-9b99-185d57c4d2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:36.876240 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6f67f-b05d-4cb9-9b99-185d57c4d2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:36.877178 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6f67f-b05d-4cb9-9b99-185d57c4d2b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:36.877490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6f67f-b05d-4cb9-9b99-185d57c4d2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:36.878983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6f67f-b05d-4cb9-9b99-185d57c4d2b1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:36.889743 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 189/374] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:36.896090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6f67f-b05d-4cb9-9b99-185d57c4d2b1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:37.019099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b994b-4ebd-4e04-be61-c2fb20f4aef2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:37.043313 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6f67f-b05d-4cb9-9b99-185d57c4d2b1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:37.069171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87aa11-f0d6-443e-b357-6e055a126f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:37.071320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87aa11-f0d6-443e-b357-6e055a126f45 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:37.071320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87aa11-f0d6-443e-b357-6e055a126f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:37.071320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87aa11-f0d6-443e-b357-6e055a126f45 None admin] PATH_INFO: `/v3/groups/41e3de5fa88e4d14a42cbd73510dee66` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:37.072514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87aa11-f0d6-443e-b357-6e055a126f45 None admin] RBAC: Authorizing `identity:get_group(group_id=41e3de5fa88e4d14a42cbd73510dee66)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:37.074047 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87aa11-f0d6-443e-b357-6e055a126f45 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:37.076008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 186/375] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:36 2022] GET /identity/v3/groups/41e3de5fa88e4d14a42cbd73510dee66 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:37.078204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d5e3a-c908-498b-8af5-2b504dcc5233 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:37.078204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d5e3a-c908-498b-8af5-2b504dcc5233 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:37.078613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d5e3a-c908-498b-8af5-2b504dcc5233 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:37.080377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 187/376] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:37.086246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafca01b-bc97-4889-b108-b1adb417883c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:37.086531 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafca01b-bc97-4889-b108-b1adb417883c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:37.086862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafca01b-bc97-4889-b108-b1adb417883c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:37.090622 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154d6db-adc3-4590-aadf-adc2ab0e4533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:37.091543 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154d6db-adc3-4590-aadf-adc2ab0e4533 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:37.092214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154d6db-adc3-4590-aadf-adc2ab0e4533 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:37.092685 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154d6db-adc3-4590-aadf-adc2ab0e4533 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:37.111742 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-dafca01b-bc97-4889-b108-b1adb417883c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:37.112954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-dafca01b-bc97-4889-b108-b1adb417883c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:37.171880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:37.217459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154d6db-adc3-4590-aadf-adc2ab0e4533 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:37.218964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154d6db-adc3-4590-aadf-adc2ab0e4533 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:37.229123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 188/377] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:37.230072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafca01b-bc97-4889-b108-b1adb417883c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:37.245392 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:37.339606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafca01b-bc97-4889-b108-b1adb417883c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:37.353377 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 190/378] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:55:37.354350 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154d6db-adc3-4590-aadf-adc2ab0e4533 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:37.400760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86b099-7f62-49f3-a037-5dc6b0677a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:37.401443 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86b099-7f62-49f3-a037-5dc6b0677a45 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:37.401753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86b099-7f62-49f3-a037-5dc6b0677a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:37.402393 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86b099-7f62-49f3-a037-5dc6b0677a45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:37.405015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86b099-7f62-49f3-a037-5dc6b0677a45 None admin] RBAC: Adding query filter params (group.id=41e3de5fa88e4d14a42cbd73510dee66, role.id=78a15fc7951d4f28beb57f54dace457d, scope.project.id=62e3ae8270a64809a9778530c2b3b7f0) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:37.405505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86b099-7f62-49f3-a037-5dc6b0677a45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:37.407571 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86b099-7f62-49f3-a037-5dc6b0677a45 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:37.414916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 189/379] 10.222.0.79 () {56 vars in 1376 bytes} [Mon Jul 4 09:55:37 2022] GET /identity/v3/role_assignments?group.id=41e3de5fa88e4d14a42cbd73510dee66&scope.project.id=62e3ae8270a64809a9778530c2b3b7f0&role.id=78a15fc7951d4f28beb57f54dace457d => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:37.415769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86b099-7f62-49f3-a037-5dc6b0677a45 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:37.475114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:37.541882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:37.552579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154d6db-adc3-4590-aadf-adc2ab0e4533 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:37.588177 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86b099-7f62-49f3-a037-5dc6b0677a45 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:37.629823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78041f66-adab-4958-ad95-7206aa022e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:37.630567 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78041f66-adab-4958-ad95-7206aa022e43 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:37.630850 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78041f66-adab-4958-ad95-7206aa022e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:37.631127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78041f66-adab-4958-ad95-7206aa022e43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:37.636102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb5211-923e-4e04-ac5d-7f1c6478dddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:37.636991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb5211-923e-4e04-ac5d-7f1c6478dddf None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:37.637269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb5211-923e-4e04-ac5d-7f1c6478dddf None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:37.637541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb5211-923e-4e04-ac5d-7f1c6478dddf None admin] PATH_INFO: `/v3/projects/ca4e68f3b1bd4167b76136fe537c9f10` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:37.640904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb5211-923e-4e04-ac5d-7f1c6478dddf None admin] RBAC: Authorizing `identity:get_project(project_id=ca4e68f3b1bd4167b76136fe537c9f10)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:37.642744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb5211-923e-4e04-ac5d-7f1c6478dddf None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:37.644812 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 190/380] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:37 2022] GET /identity/v3/projects/ca4e68f3b1bd4167b76136fe537c9f10 => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:37.649164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78041f66-adab-4958-ad95-7206aa022e43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:37.653860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb5211-923e-4e04-ac5d-7f1c6478dddf None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-78041f66-adab-4958-ad95-7206aa022e43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:37.658683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:37.659570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:37.660353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:37.660353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:37.660353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:37.660353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:37.660353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:37.660353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:37.660353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:37.660353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:37.660353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:37.660353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:37.660353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:37.660353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:37.660353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 191/381] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:37.667400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78041f66-adab-4958-ad95-7206aa022e43 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:37.850342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb5211-923e-4e04-ac5d-7f1c6478dddf None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:37.866381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78041f66-adab-4958-ad95-7206aa022e43 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:37.915488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:37.917678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68009a-aa79-4d43-bdac-0ae7f5c4440c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:37.918624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68009a-aa79-4d43-bdac-0ae7f5c4440c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:37.918916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68009a-aa79-4d43-bdac-0ae7f5c4440c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:37.919233 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68009a-aa79-4d43-bdac-0ae7f5c4440c None admin] PATH_INFO: `/v3/groups/6ef6458bfebf408bbfc4441465944997` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:37.932416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68009a-aa79-4d43-bdac-0ae7f5c4440c None admin] RBAC: Authorizing `identity:get_group(group_id=6ef6458bfebf408bbfc4441465944997)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:37.933949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68009a-aa79-4d43-bdac-0ae7f5c4440c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:37.940029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34454ada-aa67-437c-9129-197a1e9c8b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:37.940713 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34454ada-aa67-437c-9129-197a1e9c8b32 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:37.941001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34454ada-aa67-437c-9129-197a1e9c8b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:37.941282 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34454ada-aa67-437c-9129-197a1e9c8b32 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:37.945909 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 191/382] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:37 2022] GET /identity/v3/groups/6ef6458bfebf408bbfc4441465944997 => generated 229 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:37.946418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34454ada-aa67-437c-9129-197a1e9c8b32 None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:37.946742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34454ada-aa67-437c-9129-197a1e9c8b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:37.948254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34454ada-aa67-437c-9129-197a1e9c8b32 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:37.951460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68009a-aa79-4d43-bdac-0ae7f5c4440c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:37.965977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:37.969606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 192/383] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:37.977851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34454ada-aa67-437c-9129-197a1e9c8b32 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:38.120650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34454ada-aa67-437c-9129-197a1e9c8b32 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:38.169098 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cb49c-6d32-4180-8667-aa63340e1852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:38.169918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cb49c-6d32-4180-8667-aa63340e1852 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:38.170205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cb49c-6d32-4180-8667-aa63340e1852 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:38.170481 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cb49c-6d32-4180-8667-aa63340e1852 None admin] PATH_INFO: `/v3/groups/41e3de5fa88e4d14a42cbd73510dee66` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:38.171803 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84cb49c-6d32-4180-8667-aa63340e1852 None admin] RBAC: Authorizing `identity:get_group(group_id=41e3de5fa88e4d14a42cbd73510dee66)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:38.175222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84cb49c-6d32-4180-8667-aa63340e1852 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:38.177240 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 193/384] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:37 2022] GET /identity/v3/groups/41e3de5fa88e4d14a42cbd73510dee66 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:38.186785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cb49c-6d32-4180-8667-aa63340e1852 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:38.198045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68009a-aa79-4d43-bdac-0ae7f5c4440c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:38.266558 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43998e-478d-459c-982a-8ac17ca00e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:38.267230 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43998e-478d-459c-982a-8ac17ca00e25 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:38.267671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43998e-478d-459c-982a-8ac17ca00e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:38.267955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43998e-478d-459c-982a-8ac17ca00e25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:38.268800 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43998e-478d-459c-982a-8ac17ca00e25 None admin] RBAC: Adding query filter params (group.id=6ef6458bfebf408bbfc4441465944997, role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=ca4e68f3b1bd4167b76136fe537c9f10) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:38.269114 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43998e-478d-459c-982a-8ac17ca00e25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:38.272288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43998e-478d-459c-982a-8ac17ca00e25 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:38.284477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 192/385] 10.222.0.79 () {56 vars in 1376 bytes} [Mon Jul 4 09:55:37 2022] GET /identity/v3/role_assignments?group.id=6ef6458bfebf408bbfc4441465944997&scope.project.id=ca4e68f3b1bd4167b76136fe537c9f10&role.id=a3de9df253294d069dc1867fa26600c3 => generated 260 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:38.289907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:38.307650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cd212-2096-411b-b7a9-7203b8307720 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:38.307940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cd212-2096-411b-b7a9-7203b8307720 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:38.308410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cd212-2096-411b-b7a9-7203b8307720 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:38.310131 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 193/386] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:38.323754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bd9cc-2340-4836-a017-f19d7b711cda None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:38.324442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bd9cc-2340-4836-a017-f19d7b711cda None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:38.324903 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bd9cc-2340-4836-a017-f19d7b711cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:38.357406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-bb6bd9cc-2340-4836-a017-f19d7b711cda None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:38.358672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-bb6bd9cc-2340-4836-a017-f19d7b711cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:38.377549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cb49c-6d32-4180-8667-aa63340e1852 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:38.446141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e092c5-9f1c-47b6-95f0-426412c772bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:38.446956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e092c5-9f1c-47b6-95f0-426412c772bf None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:38.447243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e092c5-9f1c-47b6-95f0-426412c772bf None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:38.447526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e092c5-9f1c-47b6-95f0-426412c772bf None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:38.449788 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e092c5-9f1c-47b6-95f0-426412c772bf None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:38.451270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e092c5-9f1c-47b6-95f0-426412c772bf None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:38.464409 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 194/387] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:38 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:38.466859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e092c5-9f1c-47b6-95f0-426412c772bf None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:38.501135 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 194/388] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:38.524744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bd9cc-2340-4836-a017-f19d7b711cda None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:38.527162 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:38.601154 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e092c5-9f1c-47b6-95f0-426412c772bf None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:38.635693 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:38.640403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bd9cc-2340-4836-a017-f19d7b711cda None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:38.647397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972be541-5a8e-4d5f-b137-0d6cbd56f005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:38.648357 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972be541-5a8e-4d5f-b137-0d6cbd56f005 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:38.648643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972be541-5a8e-4d5f-b137-0d6cbd56f005 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:38.648920 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972be541-5a8e-4d5f-b137-0d6cbd56f005 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd/groups/41e3de5fa88e4d14a42cbd73510dee66/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:38.652976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972be541-5a8e-4d5f-b137-0d6cbd56f005 None admin] RBAC: Authorizing `identity:create_grant(project_id=27186cff1e94461099e5d6cc062b1abd, group_id=41e3de5fa88e4d14a42cbd73510dee66, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:38.654612 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972be541-5a8e-4d5f-b137-0d6cbd56f005 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:38.674643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 195/389] 10.222.0.79 () {58 vars in 1442 bytes} [Mon Jul 4 09:55:38 2022] PUT /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd/groups/41e3de5fa88e4d14a42cbd73510dee66/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:55:38.686529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fdc99-3b47-44d2-90c9-7a0e41066de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:38.687822 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fdc99-3b47-44d2-90c9-7a0e41066de2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:38.688370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fdc99-3b47-44d2-90c9-7a0e41066de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:38.688848 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fdc99-3b47-44d2-90c9-7a0e41066de2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:38.704080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1fdc99-3b47-44d2-90c9-7a0e41066de2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-3e1fdc99-3b47-44d2-90c9-7a0e41066de2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:38.712801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:38.713852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:38.714674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:38.714674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:38.714674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:38.714674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:38.714674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:38.714674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:38.714674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:38.714674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:38.714674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:38.714674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:38.714674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:38.714674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:38.717671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 195/390] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:55:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:38.724011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972be541-5a8e-4d5f-b137-0d6cbd56f005 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:38.852936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972be541-5a8e-4d5f-b137-0d6cbd56f005 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:38.892421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c01a9-ac10-4e7b-bd24-566856cfb23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:38.893129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c01a9-ac10-4e7b-bd24-566856cfb23c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:38.893566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c01a9-ac10-4e7b-bd24-566856cfb23c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:38.893880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c01a9-ac10-4e7b-bd24-566856cfb23c None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:38.894717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c01a9-ac10-4e7b-bd24-566856cfb23c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:38.895326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c01a9-ac10-4e7b-bd24-566856cfb23c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:38.896765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c01a9-ac10-4e7b-bd24-566856cfb23c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:38.906451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 196/391] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:38.912447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fdc99-3b47-44d2-90c9-7a0e41066de2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:38.988137 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:39.042167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fdc99-3b47-44d2-90c9-7a0e41066de2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:39.079885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac37ca-4daf-4f73-9af8-0f9c68ba4420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:39.080763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac37ca-4daf-4f73-9af8-0f9c68ba4420 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:39.081044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac37ca-4daf-4f73-9af8-0f9c68ba4420 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:39.081318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac37ca-4daf-4f73-9af8-0f9c68ba4420 None admin] PATH_INFO: `/v3/groups/41e3de5fa88e4d14a42cbd73510dee66` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:39.082556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac37ca-4daf-4f73-9af8-0f9c68ba4420 None admin] RBAC: Authorizing `identity:get_group(group_id=41e3de5fa88e4d14a42cbd73510dee66)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:39.084036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac37ca-4daf-4f73-9af8-0f9c68ba4420 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:39.085703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 196/392] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:38 2022] GET /identity/v3/groups/41e3de5fa88e4d14a42cbd73510dee66 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:39.092156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c01a9-ac10-4e7b-bd24-566856cfb23c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:39.159787 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:39.231675 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c01a9-ac10-4e7b-bd24-566856cfb23c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:39.264539 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cd4c6-1ff1-4340-bac3-f75d6914bc84 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:39.265501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cd4c6-1ff1-4340-bac3-f75d6914bc84 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:39.266051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cd4c6-1ff1-4340-bac3-f75d6914bc84 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:39.267895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 197/393] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:39.274606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8ada6-00d8-49fe-a30b-0c0244e21394 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:39.275165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8ada6-00d8-49fe-a30b-0c0244e21394 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:39.275665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8ada6-00d8-49fe-a30b-0c0244e21394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:39.275775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09385d1-1b12-462c-a716-4233855fed30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:39.276590 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09385d1-1b12-462c-a716-4233855fed30 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:39.276869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09385d1-1b12-462c-a716-4233855fed30 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:39.277144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09385d1-1b12-462c-a716-4233855fed30 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:39.282455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09385d1-1b12-462c-a716-4233855fed30 None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:39.284124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09385d1-1b12-462c-a716-4233855fed30 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:39.286299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 197/394] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:39 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:39.292245 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09385d1-1b12-462c-a716-4233855fed30 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:39.299281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c4a8ada6-00d8-49fe-a30b-0c0244e21394 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:39.300939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c4a8ada6-00d8-49fe-a30b-0c0244e21394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:39.386788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 198/395] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:39.421463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8ada6-00d8-49fe-a30b-0c0244e21394 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:39.523956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09385d1-1b12-462c-a716-4233855fed30 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:39.538013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:39.551720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:39.593106 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a62e6-b2bb-4054-a9b6-ba756312c900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:39.594023 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a62e6-b2bb-4054-a9b6-ba756312c900 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:39.594372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a62e6-b2bb-4054-a9b6-ba756312c900 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:39.594669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a62e6-b2bb-4054-a9b6-ba756312c900 None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd/groups/41e3de5fa88e4d14a42cbd73510dee66/roles/78a15fc7951d4f28beb57f54dace457d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:39.598836 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8ada6-00d8-49fe-a30b-0c0244e21394 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:39.601312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a62e6-b2bb-4054-a9b6-ba756312c900 None admin] RBAC: Authorizing `identity:create_grant(project_id=27186cff1e94461099e5d6cc062b1abd, group_id=41e3de5fa88e4d14a42cbd73510dee66, role_id=78a15fc7951d4f28beb57f54dace457d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:39.602800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a62e6-b2bb-4054-a9b6-ba756312c900 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:39.637810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 198/396] 10.222.0.79 () {58 vars in 1442 bytes} [Mon Jul 4 09:55:39 2022] PUT /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd/groups/41e3de5fa88e4d14a42cbd73510dee66/roles/78a15fc7951d4f28beb57f54dace457d => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:55:39.644007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07c55d-d066-4936-8261-b0834e2e2059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:39.644835 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07c55d-d066-4936-8261-b0834e2e2059 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:39.645115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07c55d-d066-4936-8261-b0834e2e2059 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:39.645391 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07c55d-d066-4936-8261-b0834e2e2059 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:39.658726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07c55d-d066-4936-8261-b0834e2e2059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-4a07c55d-d066-4936-8261-b0834e2e2059 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:39.666965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:39.667916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:39.669262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:39.669262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:39.669262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:39.669262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:39.669262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:39.669262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:39.669262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:39.669262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:39.669262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:39.669262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:39.669262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:39.669262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:39.669262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 199/397] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:39.676597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a62e6-b2bb-4054-a9b6-ba756312c900 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:39.853471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a62e6-b2bb-4054-a9b6-ba756312c900 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:39.896258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4cafa-ac39-42a4-b110-a6dac006eedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:39.896952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4cafa-ac39-42a4-b110-a6dac006eedc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:39.897235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4cafa-ac39-42a4-b110-a6dac006eedc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:39.897514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4cafa-ac39-42a4-b110-a6dac006eedc None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:39.898302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba4cafa-ac39-42a4-b110-a6dac006eedc None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:39.898599 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba4cafa-ac39-42a4-b110-a6dac006eedc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:39.900004 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba4cafa-ac39-42a4-b110-a6dac006eedc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:39.909710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 199/398] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:39.915368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07c55d-d066-4936-8261-b0834e2e2059 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:40.040072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc502d-ff6f-4735-a188-762092fbd952 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:40.040373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc502d-ff6f-4735-a188-762092fbd952 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:40.040659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc502d-ff6f-4735-a188-762092fbd952 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:40.042358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 200/399] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:40.050757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f08fd2-866e-434f-bab5-ecbe76b8ae71 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:40.051055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f08fd2-866e-434f-bab5-ecbe76b8ae71 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:40.051335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f08fd2-866e-434f-bab5-ecbe76b8ae71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:40.075840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07c55d-d066-4936-8261-b0834e2e2059 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:40.078727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-59f08fd2-866e-434f-bab5-ecbe76b8ae71 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:40.087146 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-59f08fd2-866e-434f-bab5-ecbe76b8ae71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:40.112829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:40.142204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:40.146631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69348faf-6cfb-4057-9cb9-85491ecea2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:40.150697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69348faf-6cfb-4057-9cb9-85491ecea2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:40.151262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69348faf-6cfb-4057-9cb9-85491ecea2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:40.151754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69348faf-6cfb-4057-9cb9-85491ecea2f0 None admin] PATH_INFO: `/v3/groups/41e3de5fa88e4d14a42cbd73510dee66` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:40.153246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69348faf-6cfb-4057-9cb9-85491ecea2f0 None admin] RBAC: Authorizing `identity:get_group(group_id=41e3de5fa88e4d14a42cbd73510dee66)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:40.154971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69348faf-6cfb-4057-9cb9-85491ecea2f0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:40.161160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 200/400] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:39 2022] GET /identity/v3/groups/41e3de5fa88e4d14a42cbd73510dee66 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:40.166132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69348faf-6cfb-4057-9cb9-85491ecea2f0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:40.244443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 201/401] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:40.284478 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69348faf-6cfb-4057-9cb9-85491ecea2f0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:40.286648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f08fd2-866e-434f-bab5-ecbe76b8ae71 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:40.329244 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfd0d3-d488-407b-84f0-4da103f5495c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:40.330967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfd0d3-d488-407b-84f0-4da103f5495c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:40.331540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfd0d3-d488-407b-84f0-4da103f5495c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:40.332020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfd0d3-d488-407b-84f0-4da103f5495c None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:40.334611 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfd0d3-d488-407b-84f0-4da103f5495c None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:40.336450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfd0d3-d488-407b-84f0-4da103f5495c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:40.342291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 201/402] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:40 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:40.345351 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfd0d3-d488-407b-84f0-4da103f5495c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:40.422302 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:40.451855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f08fd2-866e-434f-bab5-ecbe76b8ae71 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:40.502146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfd0d3-d488-407b-84f0-4da103f5495c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:40.527403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e0926-39df-4c93-afd6-40ed5fb8f9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:40.528602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e0926-39df-4c93-afd6-40ed5fb8f9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:40.529228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e0926-39df-4c93-afd6-40ed5fb8f9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:40.530555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e0926-39df-4c93-afd6-40ed5fb8f9d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:40.556658 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e0926-39df-4c93-afd6-40ed5fb8f9d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-f43e0926-39df-4c93-afd6-40ed5fb8f9d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:40.567051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:40.567941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:40.568726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:40.568726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:40.568726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:40.568726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:40.568726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:40.568726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:40.568726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:40.568726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:40.568726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:40.568726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:40.568726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:40.568726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:40.575441 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39656210-5e11-4750-bc72-f5e28506cf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:40.576562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39656210-5e11-4750-bc72-f5e28506cf57 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:40.576990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39656210-5e11-4750-bc72-f5e28506cf57 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:40.577421 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39656210-5e11-4750-bc72-f5e28506cf57 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0/groups/41e3de5fa88e4d14a42cbd73510dee66/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:40.584116 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 202/403] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:55:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:40.585307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e0926-39df-4c93-afd6-40ed5fb8f9d3 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:40.585895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39656210-5e11-4750-bc72-f5e28506cf57 None admin] RBAC: Authorizing `identity:create_grant(project_id=62e3ae8270a64809a9778530c2b3b7f0, group_id=41e3de5fa88e4d14a42cbd73510dee66, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:40.587539 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39656210-5e11-4750-bc72-f5e28506cf57 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:40.608910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 202/404] 10.222.0.79 () {58 vars in 1442 bytes} [Mon Jul 4 09:55:40 2022] PUT /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0/groups/41e3de5fa88e4d14a42cbd73510dee66/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:55:40.639907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:40.740255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e0926-39df-4c93-afd6-40ed5fb8f9d3 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:40.789038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe42be3-938c-4629-b081-7b4327eadd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:40.789785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe42be3-938c-4629-b081-7b4327eadd1f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:40.790067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe42be3-938c-4629-b081-7b4327eadd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:40.790338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe42be3-938c-4629-b081-7b4327eadd1f None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:40.791104 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe42be3-938c-4629-b081-7b4327eadd1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:40.791410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe42be3-938c-4629-b081-7b4327eadd1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:40.792835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe42be3-938c-4629-b081-7b4327eadd1f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:40.802816 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 203/405] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:40.808738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39656210-5e11-4750-bc72-f5e28506cf57 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:40.917028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39656210-5e11-4750-bc72-f5e28506cf57 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:40.933820 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:40.952781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce46638-48fd-4d62-b788-0fbca82c9fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:40.953725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce46638-48fd-4d62-b788-0fbca82c9fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:40.954090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce46638-48fd-4d62-b788-0fbca82c9fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:40.954465 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce46638-48fd-4d62-b788-0fbca82c9fe9 None admin] PATH_INFO: `/v3/groups/41e3de5fa88e4d14a42cbd73510dee66` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:40.956123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce46638-48fd-4d62-b788-0fbca82c9fe9 None admin] RBAC: Authorizing `identity:get_group(group_id=41e3de5fa88e4d14a42cbd73510dee66)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:40.957899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce46638-48fd-4d62-b788-0fbca82c9fe9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:40.959908 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 203/406] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:40 2022] GET /identity/v3/groups/41e3de5fa88e4d14a42cbd73510dee66 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:40.965474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe42be3-938c-4629-b081-7b4327eadd1f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:40.986784 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dcbfe-74a3-4447-8abe-188b478e4654 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:40.987114 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dcbfe-74a3-4447-8abe-188b478e4654 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:40.987420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dcbfe-74a3-4447-8abe-188b478e4654 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:40.988933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 204/407] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:40.994053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92edeaf4-01f0-4261-a3cc-471716c6f9c1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:40.994274 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92edeaf4-01f0-4261-a3cc-471716c6f9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:40.994552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92edeaf4-01f0-4261-a3cc-471716c6f9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:41.011434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-92edeaf4-01f0-4261-a3cc-471716c6f9c1 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:41.012610 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-92edeaf4-01f0-4261-a3cc-471716c6f9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:41.139752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 205/408] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:41.150495 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:41.161418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe42be3-938c-4629-b081-7b4327eadd1f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:41.194904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92edeaf4-01f0-4261-a3cc-471716c6f9c1 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:41.223634 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8eee09-8e45-485d-b299-34c6db9ecd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:41.224558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8eee09-8e45-485d-b299-34c6db9ecd34 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:41.224841 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8eee09-8e45-485d-b299-34c6db9ecd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:41.225114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8eee09-8e45-485d-b299-34c6db9ecd34 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:41.229379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8eee09-8e45-485d-b299-34c6db9ecd34 None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:41.230870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8eee09-8e45-485d-b299-34c6db9ecd34 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:41.233140 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 204/409] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:40 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:41.238882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8eee09-8e45-485d-b299-34c6db9ecd34 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:41.333067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92edeaf4-01f0-4261-a3cc-471716c6f9c1 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:41.361350 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:41.388475 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3538fe-0c7c-438e-a5d7-3dc56ff72457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:41.389280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3538fe-0c7c-438e-a5d7-3dc56ff72457 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:41.389560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3538fe-0c7c-438e-a5d7-3dc56ff72457 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:41.389860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3538fe-0c7c-438e-a5d7-3dc56ff72457 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:41.411450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3538fe-0c7c-438e-a5d7-3dc56ff72457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-5d3538fe-0c7c-438e-a5d7-3dc56ff72457 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:41.419236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:41.420103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:41.420885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:41.420885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:41.420885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:41.420885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:41.420885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:41.420885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:41.420885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:41.420885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:41.420885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:41.420885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:41.420885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:41.420885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:41.420885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 206/410] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:55:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:41.428001 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3538fe-0c7c-438e-a5d7-3dc56ff72457 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:41.439420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8eee09-8e45-485d-b299-34c6db9ecd34 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:41.500576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa287b3-7188-42c6-b1a8-5954bf894fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:41.501607 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa287b3-7188-42c6-b1a8-5954bf894fd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:41.501913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa287b3-7188-42c6-b1a8-5954bf894fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:41.502191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa287b3-7188-42c6-b1a8-5954bf894fd0 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0/groups/41e3de5fa88e4d14a42cbd73510dee66/roles/78a15fc7951d4f28beb57f54dace457d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:41.506262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa287b3-7188-42c6-b1a8-5954bf894fd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=62e3ae8270a64809a9778530c2b3b7f0, group_id=41e3de5fa88e4d14a42cbd73510dee66, role_id=78a15fc7951d4f28beb57f54dace457d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:41.507721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa287b3-7188-42c6-b1a8-5954bf894fd0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:41.529834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 205/411] 10.222.0.79 () {58 vars in 1442 bytes} [Mon Jul 4 09:55:41 2022] PUT /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0/groups/41e3de5fa88e4d14a42cbd73510dee66/roles/78a15fc7951d4f28beb57f54dace457d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:55:41.584384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a899b0-c5c0-48cb-a018-be4a3b9da508 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:41.585006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a899b0-c5c0-48cb-a018-be4a3b9da508 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:41.585491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a899b0-c5c0-48cb-a018-be4a3b9da508 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:41.587442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 206/412] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:41.593203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80c3a2-5e12-4b40-a4d2-efb31229fb56 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:41.593792 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80c3a2-5e12-4b40-a4d2-efb31229fb56 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:41.594014 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3538fe-0c7c-438e-a5d7-3dc56ff72457 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:41.594561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80c3a2-5e12-4b40-a4d2-efb31229fb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:41.613655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9b80c3a2-5e12-4b40-a4d2-efb31229fb56 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:41.615248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9b80c3a2-5e12-4b40-a4d2-efb31229fb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:41.630765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:41.667363 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c9a49-a289-48e8-b1cb-5f0216399c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:41.669648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c9a49-a289-48e8-b1cb-5f0216399c53 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:41.671048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c9a49-a289-48e8-b1cb-5f0216399c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:41.671569 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c9a49-a289-48e8-b1cb-5f0216399c53 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:41.673136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c9a49-a289-48e8-b1cb-5f0216399c53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:41.675056 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c9a49-a289-48e8-b1cb-5f0216399c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:41.677789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c9a49-a289-48e8-b1cb-5f0216399c53 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:41.692267 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 207/413] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:41.698287 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c9a49-a289-48e8-b1cb-5f0216399c53 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:41.735568 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 207/414] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:41.762635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:41.775357 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80c3a2-5e12-4b40-a4d2-efb31229fb56 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:41.847797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c9a49-a289-48e8-b1cb-5f0216399c53 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:41.904533 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57637e9f-ceb0-412b-b6a6-ed8a4f4ab1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:41.905410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57637e9f-ceb0-412b-b6a6-ed8a4f4ab1fb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:41.905725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57637e9f-ceb0-412b-b6a6-ed8a4f4ab1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:41.905999 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57637e9f-ceb0-412b-b6a6-ed8a4f4ab1fb None admin] PATH_INFO: `/v3/groups/6ef6458bfebf408bbfc4441465944997` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:41.908280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57637e9f-ceb0-412b-b6a6-ed8a4f4ab1fb None admin] RBAC: Authorizing `identity:get_group(group_id=6ef6458bfebf408bbfc4441465944997)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:41.909798 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57637e9f-ceb0-412b-b6a6-ed8a4f4ab1fb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:41.913746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 208/415] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:41 2022] GET /identity/v3/groups/6ef6458bfebf408bbfc4441465944997 => generated 229 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:41.919296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57637e9f-ceb0-412b-b6a6-ed8a4f4ab1fb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:41.921948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80c3a2-5e12-4b40-a4d2-efb31229fb56 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:41.989171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bc509-6c54-418f-98b2-65d4379c0806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:41.990027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bc509-6c54-418f-98b2-65d4379c0806 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:41.990313 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bc509-6c54-418f-98b2-65d4379c0806 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:41.990599 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bc509-6c54-418f-98b2-65d4379c0806 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:42.012179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bc509-6c54-418f-98b2-65d4379c0806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-647bc509-6c54-418f-98b2-65d4379c0806 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:42.020137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:42.021001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:42.021835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:42.021835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:42.021835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:42.021835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:42.021835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:42.021835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:42.021835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:42.021835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:42.021835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:42.021835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:42.021835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:42.021835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:42.021835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 208/416] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:42.033690 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bc509-6c54-418f-98b2-65d4379c0806 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:42.054727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:42.119656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57637e9f-ceb0-412b-b6a6-ed8a4f4ab1fb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:42.130242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:42.159431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bc509-6c54-418f-98b2-65d4379c0806 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:42.201486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646012d-2172-43be-b7f3-44b8adafbb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:42.202193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646012d-2172-43be-b7f3-44b8adafbb69 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:42.202474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646012d-2172-43be-b7f3-44b8adafbb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:42.202751 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646012d-2172-43be-b7f3-44b8adafbb69 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:42.203509 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646012d-2172-43be-b7f3-44b8adafbb69 None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:42.203972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646012d-2172-43be-b7f3-44b8adafbb69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:42.205768 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646012d-2172-43be-b7f3-44b8adafbb69 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:42.215673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 209/417] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:42.221562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646012d-2172-43be-b7f3-44b8adafbb69 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:42.226125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25473e57-2ddf-4235-af63-ee7e06d70285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:42.227018 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25473e57-2ddf-4235-af63-ee7e06d70285 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:42.227298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25473e57-2ddf-4235-af63-ee7e06d70285 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:42.227574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25473e57-2ddf-4235-af63-ee7e06d70285 None admin] PATH_INFO: `/v3/projects/ca4e68f3b1bd4167b76136fe537c9f10` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:42.235436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25473e57-2ddf-4235-af63-ee7e06d70285 None admin] RBAC: Authorizing `identity:get_project(project_id=ca4e68f3b1bd4167b76136fe537c9f10)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:42.236945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25473e57-2ddf-4235-af63-ee7e06d70285 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:42.239043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 209/418] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:41 2022] GET /identity/v3/projects/ca4e68f3b1bd4167b76136fe537c9f10 => generated 344 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:42.249116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25473e57-2ddf-4235-af63-ee7e06d70285 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:42.301881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:42.335582 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646012d-2172-43be-b7f3-44b8adafbb69 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:42.376994 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a17c8-e85a-49ad-bcca-b6bb104d58bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:42.377573 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a17c8-e85a-49ad-bcca-b6bb104d58bc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:42.377886 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a17c8-e85a-49ad-bcca-b6bb104d58bc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:42.378162 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a17c8-e85a-49ad-bcca-b6bb104d58bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:42.385554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25473e57-2ddf-4235-af63-ee7e06d70285 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:42.430402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c2e1d-788d-43d4-b639-e98e09ad745c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:42.431947 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c2e1d-788d-43d4-b639-e98e09ad745c None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:42.432692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c2e1d-788d-43d4-b639-e98e09ad745c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:42.433444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c2e1d-788d-43d4-b639-e98e09ad745c None admin] PATH_INFO: `/v3/projects/ca4e68f3b1bd4167b76136fe537c9f10/groups/6ef6458bfebf408bbfc4441465944997/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:42.436224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c2e1d-788d-43d4-b639-e98e09ad745c None admin] RBAC: Authorizing `identity:create_grant(project_id=ca4e68f3b1bd4167b76136fe537c9f10, group_id=6ef6458bfebf408bbfc4441465944997, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:42.438603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c2e1d-788d-43d4-b639-e98e09ad745c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:42.459154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 210/419] 10.222.0.79 () {58 vars in 1442 bytes} [Mon Jul 4 09:55:42 2022] PUT /identity/v3/projects/ca4e68f3b1bd4167b76136fe537c9f10/groups/6ef6458bfebf408bbfc4441465944997/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:55:42.463702 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79221c9c-58ef-4f88-bbc8-49783c290f53 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:42.464418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79221c9c-58ef-4f88-bbc8-49783c290f53 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:42.464869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79221c9c-58ef-4f88-bbc8-49783c290f53 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:42.466570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a17c8-e85a-49ad-bcca-b6bb104d58bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:42.467622 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a17c8-e85a-49ad-bcca-b6bb104d58bc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:42.473837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 211/420] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:42.476333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef10a65-f34a-45a3-ae28-0a515e70b4b4 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:42.476813 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef10a65-f34a-45a3-ae28-0a515e70b4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:42.477282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef10a65-f34a-45a3-ae28-0a515e70b4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:42.496946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-eef10a65-f34a-45a3-ae28-0a515e70b4b4 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:42.498740 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-eef10a65-f34a-45a3-ae28-0a515e70b4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:42.529349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:42.549196 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:42.581387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 212/421] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:42.587249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 210/422] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:42.593057 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef10a65-f34a-45a3-ae28-0a515e70b4b4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:42.610236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a17c8-e85a-49ad-bcca-b6bb104d58bc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:42.723026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a17c8-e85a-49ad-bcca-b6bb104d58bc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:42.732457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef10a65-f34a-45a3-ae28-0a515e70b4b4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:42.770254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d69db-02c6-4562-bf06-7e7152d40459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:42.771097 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d69db-02c6-4562-bf06-7e7152d40459 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:42.771376 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d69db-02c6-4562-bf06-7e7152d40459 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:42.777999 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d69db-02c6-4562-bf06-7e7152d40459 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:42.798284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d69db-02c6-4562-bf06-7e7152d40459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-2f0d69db-02c6-4562-bf06-7e7152d40459 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:42.812804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:42.814034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:42.815033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:42.815033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:42.815033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:42.815033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:42.815033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:42.815033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:42.815033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:42.815033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:42.815033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:42.815033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:42.815033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:42.815033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:42.819491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 211/423] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:55:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:42.820253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e9f64-531d-4985-be84-df1622122eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:42.821307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e9f64-531d-4985-be84-df1622122eeb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:42.821758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e9f64-531d-4985-be84-df1622122eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:42.822157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e9f64-531d-4985-be84-df1622122eeb None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:42.829706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d69db-02c6-4562-bf06-7e7152d40459 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:42.830692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e9f64-531d-4985-be84-df1622122eeb None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:42.832432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e9f64-531d-4985-be84-df1622122eeb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:42.834544 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 213/424] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:42 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:42.839946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e9f64-531d-4985-be84-df1622122eeb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:42.865974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:42.895583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:42.999854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d69db-02c6-4562-bf06-7e7152d40459 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:43.000352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e9f64-531d-4985-be84-df1622122eeb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:43.061257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74247d3-9fd8-4604-9e18-e828635882fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:43.062162 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74247d3-9fd8-4604-9e18-e828635882fd None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:43.062447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74247d3-9fd8-4604-9e18-e828635882fd None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:43.062726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74247d3-9fd8-4604-9e18-e828635882fd None admin] PATH_INFO: `/v3/groups/41e3de5fa88e4d14a42cbd73510dee66` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:43.063984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216d7a4-86a5-4c7f-a836-1b384dddd36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:43.064664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216d7a4-86a5-4c7f-a836-1b384dddd36f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:43.064941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216d7a4-86a5-4c7f-a836-1b384dddd36f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:43.065214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216d7a4-86a5-4c7f-a836-1b384dddd36f None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:43.066001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216d7a4-86a5-4c7f-a836-1b384dddd36f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:43.066312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216d7a4-86a5-4c7f-a836-1b384dddd36f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:43.067728 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216d7a4-86a5-4c7f-a836-1b384dddd36f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:43.070122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74247d3-9fd8-4604-9e18-e828635882fd None admin] RBAC: Authorizing `identity:get_group(group_id=41e3de5fa88e4d14a42cbd73510dee66)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:43.071600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74247d3-9fd8-4604-9e18-e828635882fd None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:43.073340 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 214/425] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:42 2022] GET /identity/v3/groups/41e3de5fa88e4d14a42cbd73510dee66 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:43.079788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74247d3-9fd8-4604-9e18-e828635882fd None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:43.095221 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 212/426] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:43.096156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216d7a4-86a5-4c7f-a836-1b384dddd36f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:43.226388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:43.234337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216d7a4-86a5-4c7f-a836-1b384dddd36f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:43.249685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74247d3-9fd8-4604-9e18-e828635882fd None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:43.258071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:43.270196 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cbd43-f0a6-46b8-86cd-3d289b19f979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:43.271008 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cbd43-f0a6-46b8-86cd-3d289b19f979 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:43.271395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cbd43-f0a6-46b8-86cd-3d289b19f979 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:43.271803 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cbd43-f0a6-46b8-86cd-3d289b19f979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:43.287454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960284e-f8f2-44d1-95f5-4f269103501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:43.288477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960284e-f8f2-44d1-95f5-4f269103501d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:43.288924 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960284e-f8f2-44d1-95f5-4f269103501d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:43.289747 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960284e-f8f2-44d1-95f5-4f269103501d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:43.291001 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960284e-f8f2-44d1-95f5-4f269103501d None admin] RBAC: Adding query filter params (group.id=41e3de5fa88e4d14a42cbd73510dee66, role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d, scope.project.id=27186cff1e94461099e5d6cc062b1abd) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:43.291465 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960284e-f8f2-44d1-95f5-4f269103501d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:43.293515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960284e-f8f2-44d1-95f5-4f269103501d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:43.301579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 215/427] 10.222.0.79 () {56 vars in 1376 bytes} [Mon Jul 4 09:55:43 2022] GET /identity/v3/role_assignments?group.id=41e3de5fa88e4d14a42cbd73510dee66&scope.project.id=27186cff1e94461099e5d6cc062b1abd&role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 611 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:43.303476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dfeeb-e62d-42b5-aae9-5f94320d43e0 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:43.304136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dfeeb-e62d-42b5-aae9-5f94320d43e0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:43.304657 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dfeeb-e62d-42b5-aae9-5f94320d43e0 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:43.306953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 216/428] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:43.317682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea0553-4f1d-410c-91bf-79005c378888 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:43.317682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea0553-4f1d-410c-91bf-79005c378888 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:43.317682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea0553-4f1d-410c-91bf-79005c378888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:43.358871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f8ea0553-4f1d-410c-91bf-79005c378888 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:43.362374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f8ea0553-4f1d-410c-91bf-79005c378888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:43.390610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cbd43-f0a6-46b8-86cd-3d289b19f979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:43.393588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cbd43-f0a6-46b8-86cd-3d289b19f979 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:43.497678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 217/429] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:43.520555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea0553-4f1d-410c-91bf-79005c378888 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:43.553560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:43.564288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 213/430] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:43.570978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cbd43-f0a6-46b8-86cd-3d289b19f979 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:43.641060 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:43.650608 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea0553-4f1d-410c-91bf-79005c378888 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:43.679293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cbd43-f0a6-46b8-86cd-3d289b19f979 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:43.688923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ddd9c-bda8-4c8c-b0f4-8132b0f5a0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:43.689797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ddd9c-bda8-4c8c-b0f4-8132b0f5a0be None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:43.690082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ddd9c-bda8-4c8c-b0f4-8132b0f5a0be None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:43.690357 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ddd9c-bda8-4c8c-b0f4-8132b0f5a0be None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:43.707141 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ddd9c-bda8-4c8c-b0f4-8132b0f5a0be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:43.720529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e529a-dd7d-4ced-8942-8867e7c1e53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d72ddd9c-bda8-4c8c-b0f4-8132b0f5a0be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:43.721260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:43.722149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:43.722940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:43.722940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:43.722940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:43.722940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:43.722940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:43.722940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:43.722940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:43.722940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:43.722940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:43.722940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:43.722940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:55:43.722940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:43.723620 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e529a-dd7d-4ced-8942-8867e7c1e53a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:43.723620 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e529a-dd7d-4ced-8942-8867e7c1e53a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:43.723620 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e529a-dd7d-4ced-8942-8867e7c1e53a None admin] PATH_INFO: `/v3/projects/27186cff1e94461099e5d6cc062b1abd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:43.723620 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e529a-dd7d-4ced-8942-8867e7c1e53a None admin] RBAC: Authorizing `identity:get_project(project_id=27186cff1e94461099e5d6cc062b1abd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:43.726733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e529a-dd7d-4ced-8942-8867e7c1e53a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:43.727006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 214/431] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:43 2022] GET /identity/v3/projects/27186cff1e94461099e5d6cc062b1abd => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:43.727284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 218/432] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:43.729003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e529a-dd7d-4ced-8942-8867e7c1e53a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:43.732956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ddd9c-bda8-4c8c-b0f4-8132b0f5a0be None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:43.794545 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:43.837586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e529a-dd7d-4ced-8942-8867e7c1e53a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:43.848745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ddd9c-bda8-4c8c-b0f4-8132b0f5a0be None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:43.873891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cc13d-0f02-4e6d-a874-a1354dd649cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:43.874578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cc13d-0f02-4e6d-a874-a1354dd649cc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:43.874857 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cc13d-0f02-4e6d-a874-a1354dd649cc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:43.875453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cc13d-0f02-4e6d-a874-a1354dd649cc None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:43.876206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cc13d-0f02-4e6d-a874-a1354dd649cc None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:43.876725 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cc13d-0f02-4e6d-a874-a1354dd649cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:43.877934 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cc13d-0f02-4e6d-a874-a1354dd649cc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:43.886939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b114615-292f-4764-bdf5-57473eabdf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:43.888117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b114615-292f-4764-bdf5-57473eabdf73 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:43.888586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b114615-292f-4764-bdf5-57473eabdf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:43.889045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b114615-292f-4764-bdf5-57473eabdf73 None admin] PATH_INFO: `/v3/groups/41e3de5fa88e4d14a42cbd73510dee66` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:43.890458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b114615-292f-4764-bdf5-57473eabdf73 None admin] RBAC: Authorizing `identity:get_group(group_id=41e3de5fa88e4d14a42cbd73510dee66)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:43.891555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 215/433] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:55:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:43.892580 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b114615-292f-4764-bdf5-57473eabdf73 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:43.892845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cc13d-0f02-4e6d-a874-a1354dd649cc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:43.895092 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 219/434] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:43 2022] GET /identity/v3/groups/41e3de5fa88e4d14a42cbd73510dee66 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:43.900373 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b114615-292f-4764-bdf5-57473eabdf73 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:43.925252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:43.997787 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cc13d-0f02-4e6d-a874-a1354dd649cc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:44.014969 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b114615-292f-4764-bdf5-57473eabdf73 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:44.021941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:44.034587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612bc97-57e9-41b5-ad53-b7bcd88adfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:44.035333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612bc97-57e9-41b5-ad53-b7bcd88adfb6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:44.035621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612bc97-57e9-41b5-ad53-b7bcd88adfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:44.035901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612bc97-57e9-41b5-ad53-b7bcd88adfb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:44.054709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3bd00-48e0-486b-8cc9-68c5f587a72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:44.055692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3bd00-48e0-486b-8cc9-68c5f587a72c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:44.056171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3bd00-48e0-486b-8cc9-68c5f587a72c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:44.056632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3bd00-48e0-486b-8cc9-68c5f587a72c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:44.057729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3bd00-48e0-486b-8cc9-68c5f587a72c None admin] RBAC: Adding query filter params (group.id=41e3de5fa88e4d14a42cbd73510dee66, role.id=78a15fc7951d4f28beb57f54dace457d, scope.project.id=27186cff1e94461099e5d6cc062b1abd) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:44.058232 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3bd00-48e0-486b-8cc9-68c5f587a72c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:44.059859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3bd00-48e0-486b-8cc9-68c5f587a72c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:44.067902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 220/435] 10.222.0.79 () {56 vars in 1376 bytes} [Mon Jul 4 09:55:43 2022] GET /identity/v3/role_assignments?group.id=41e3de5fa88e4d14a42cbd73510dee66&scope.project.id=27186cff1e94461099e5d6cc062b1abd&role.id=78a15fc7951d4f28beb57f54dace457d => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:44.120525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612bc97-57e9-41b5-ad53-b7bcd88adfb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:44.122443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612bc97-57e9-41b5-ad53-b7bcd88adfb6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:44.238221 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 216/436] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:44.244217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3bd00-48e0-486b-8cc9-68c5f587a72c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:44.259679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5de187-1f67-4c24-9226-08e5bc112186 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:44.259977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5de187-1f67-4c24-9226-08e5bc112186 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:44.260254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5de187-1f67-4c24-9226-08e5bc112186 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:44.261571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 217/437] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:44.267834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9813749-d674-46c5-aada-458d66d09e15 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:44.268120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9813749-d674-46c5-aada-458d66d09e15 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:44.268396 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9813749-d674-46c5-aada-458d66d09e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:44.286702 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a9813749-d674-46c5-aada-458d66d09e15 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:44.287897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a9813749-d674-46c5-aada-458d66d09e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:44.307621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:44.351811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:44.362430 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 218/438] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:55:44.373715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3bd00-48e0-486b-8cc9-68c5f587a72c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:44.389009 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9813749-d674-46c5-aada-458d66d09e15 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:44.410738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3a78c-f9c7-46ac-b174-c9da6bce0377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:44.412342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3a78c-f9c7-46ac-b174-c9da6bce0377 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:44.412785 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3a78c-f9c7-46ac-b174-c9da6bce0377 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:44.413067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3a78c-f9c7-46ac-b174-c9da6bce0377 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:44.414323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3a78c-f9c7-46ac-b174-c9da6bce0377 None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:44.415901 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3a78c-f9c7-46ac-b174-c9da6bce0377 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:44.421639 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 221/439] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:44 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:44.423615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3a78c-f9c7-46ac-b174-c9da6bce0377 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:44.509984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9813749-d674-46c5-aada-458d66d09e15 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:44.555849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b5157-af47-478d-bfeb-6310353e2d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:44.556752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b5157-af47-478d-bfeb-6310353e2d34 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:44.557039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b5157-af47-478d-bfeb-6310353e2d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:44.557346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b5157-af47-478d-bfeb-6310353e2d34 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:44.562695 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3a78c-f9c7-46ac-b174-c9da6bce0377 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:44.571666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b5157-af47-478d-bfeb-6310353e2d34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-3b3b5157-af47-478d-bfeb-6310353e2d34 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:44.580578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:44.581488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:44.582375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:44.582375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:44.582375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:44.582375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:44.582375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:44.582375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:44.582375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:44.582375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:44.582375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:44.582375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:44.582375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:44.582375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:44.582375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 219/440] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:55:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:44.590054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b5157-af47-478d-bfeb-6310353e2d34 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:44.607177 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:44.611374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704b38f-3d96-44d0-bdc9-e7d003cfc405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:44.612278 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704b38f-3d96-44d0-bdc9-e7d003cfc405 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:44.612556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704b38f-3d96-44d0-bdc9-e7d003cfc405 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:44.612790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704b38f-3d96-44d0-bdc9-e7d003cfc405 None admin] PATH_INFO: `/v3/groups/41e3de5fa88e4d14a42cbd73510dee66` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:44.614031 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704b38f-3d96-44d0-bdc9-e7d003cfc405 None admin] RBAC: Authorizing `identity:get_group(group_id=41e3de5fa88e4d14a42cbd73510dee66)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:44.615495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704b38f-3d96-44d0-bdc9-e7d003cfc405 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:44.617300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 222/441] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:44 2022] GET /identity/v3/groups/41e3de5fa88e4d14a42cbd73510dee66 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:44.622759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704b38f-3d96-44d0-bdc9-e7d003cfc405 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:44.646399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:44.697538 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b5157-af47-478d-bfeb-6310353e2d34 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:44.729417 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704b38f-3d96-44d0-bdc9-e7d003cfc405 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:44.733053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197f800-ecf2-457a-bcb9-12f1cb0bc298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:44.733757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197f800-ecf2-457a-bcb9-12f1cb0bc298 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:44.734039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197f800-ecf2-457a-bcb9-12f1cb0bc298 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:44.734322 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197f800-ecf2-457a-bcb9-12f1cb0bc298 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:44.735136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197f800-ecf2-457a-bcb9-12f1cb0bc298 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:44.735442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197f800-ecf2-457a-bcb9-12f1cb0bc298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:44.736845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197f800-ecf2-457a-bcb9-12f1cb0bc298 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:44.744858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 220/442] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:44.746148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1330f94-2841-4293-bfe3-990daad31373 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:44.746444 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1330f94-2841-4293-bfe3-990daad31373 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:44.746726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1330f94-2841-4293-bfe3-990daad31373 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:44.747949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 221/443] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:44.750306 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1330f94-2841-4293-bfe3-990daad31373 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:44.763428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eac015-c701-4a67-a553-8e3adbaa4433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:44.764096 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eac015-c701-4a67-a553-8e3adbaa4433 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:44.764383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eac015-c701-4a67-a553-8e3adbaa4433 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:44.764664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eac015-c701-4a67-a553-8e3adbaa4433 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:44.765495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eac015-c701-4a67-a553-8e3adbaa4433 None admin] RBAC: Adding query filter params (group.id=41e3de5fa88e4d14a42cbd73510dee66, role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d, scope.project.id=62e3ae8270a64809a9778530c2b3b7f0) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:44.765841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eac015-c701-4a67-a553-8e3adbaa4433 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:44.767329 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eac015-c701-4a67-a553-8e3adbaa4433 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:44.773241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 223/444] 10.222.0.79 () {56 vars in 1376 bytes} [Mon Jul 4 09:55:44 2022] GET /identity/v3/role_assignments?group.id=41e3de5fa88e4d14a42cbd73510dee66&scope.project.id=62e3ae8270a64809a9778530c2b3b7f0&role.id=c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:44.774428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dca3da-d71b-407e-8450-5756d1fd7d5a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:44.774720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dca3da-d71b-407e-8450-5756d1fd7d5a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:44.775021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dca3da-d71b-407e-8450-5756d1fd7d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:44.792484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d2dca3da-d71b-407e-8450-5756d1fd7d5a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:44.793768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d2dca3da-d71b-407e-8450-5756d1fd7d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:44.840641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:44.868781 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1330f94-2841-4293-bfe3-990daad31373 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:44.878050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 224/445] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:44.898085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dca3da-d71b-407e-8450-5756d1fd7d5a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:44.906862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648abe0c-6903-43e5-91fd-42457a649560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:44.907569 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648abe0c-6903-43e5-91fd-42457a649560 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:44.907853 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648abe0c-6903-43e5-91fd-42457a649560 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:44.908125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648abe0c-6903-43e5-91fd-42457a649560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:44.915440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:44.991231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648abe0c-6903-43e5-91fd-42457a649560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:44.993133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648abe0c-6903-43e5-91fd-42457a649560 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:45.023928 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dca3da-d71b-407e-8450-5756d1fd7d5a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:45.073587 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb06afb-edd6-4e98-b47a-3744c19a2362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:45.074420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb06afb-edd6-4e98-b47a-3744c19a2362 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:45.074700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb06afb-edd6-4e98-b47a-3744c19a2362 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:45.074977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb06afb-edd6-4e98-b47a-3744c19a2362 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:45.088065 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:45.088239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb06afb-edd6-4e98-b47a-3744c19a2362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-dfb06afb-edd6-4e98-b47a-3744c19a2362 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:45.097511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:45.098458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:45.099258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:45.099258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:45.099258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:45.099258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:45.099258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:45.099258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:45.099258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:45.099258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:45.099258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:45.099258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:45.099258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:55:45.099258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:45.099258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 225/446] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:55:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:45.105021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb06afb-edd6-4e98-b47a-3744c19a2362 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:45.127352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 222/447] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:45.129949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648abe0c-6903-43e5-91fd-42457a649560 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:45.187168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:45.211889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb06afb-edd6-4e98-b47a-3744c19a2362 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:45.227749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648abe0c-6903-43e5-91fd-42457a649560 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:45.247318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f488e3-259a-43f3-85a2-be47d34e30a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:45.247988 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f488e3-259a-43f3-85a2-be47d34e30a3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:45.248270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f488e3-259a-43f3-85a2-be47d34e30a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:45.248548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f488e3-259a-43f3-85a2-be47d34e30a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:45.249304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f488e3-259a-43f3-85a2-be47d34e30a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:45.249595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f488e3-259a-43f3-85a2-be47d34e30a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:45.251382 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f488e3-259a-43f3-85a2-be47d34e30a3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:45.259409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 226/448] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:45.264227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d45287-98c8-4831-9955-605b1a9cc3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:45.264227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d45287-98c8-4831-9955-605b1a9cc3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:45.264227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d45287-98c8-4831-9955-605b1a9cc3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:45.264227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d45287-98c8-4831-9955-605b1a9cc3c7 None admin] PATH_INFO: `/v3/projects/62e3ae8270a64809a9778530c2b3b7f0` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:45.264536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d45287-98c8-4831-9955-605b1a9cc3c7 None admin] RBAC: Authorizing `identity:get_project(project_id=62e3ae8270a64809a9778530c2b3b7f0)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:45.264680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d45287-98c8-4831-9955-605b1a9cc3c7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:45.265071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f488e3-259a-43f3-85a2-be47d34e30a3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:45.266463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 223/449] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:45 2022] GET /identity/v3/projects/62e3ae8270a64809a9778530c2b3b7f0 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:45.271915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d45287-98c8-4831-9955-605b1a9cc3c7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:45.313692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:45.363550 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f488e3-259a-43f3-85a2-be47d34e30a3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:45.369410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d45287-98c8-4831-9955-605b1a9cc3c7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:45.397811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f8a72-5360-4bd8-b446-53304d7f46e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:45.398462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f8a72-5360-4bd8-b446-53304d7f46e0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:45.398735 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f8a72-5360-4bd8-b446-53304d7f46e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:45.399013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f8a72-5360-4bd8-b446-53304d7f46e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:45.402548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:45.403911 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345a310-aa3b-41b3-b0aa-e127971f290f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:45.404893 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345a310-aa3b-41b3-b0aa-e127971f290f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:45.405319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345a310-aa3b-41b3-b0aa-e127971f290f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:45.405599 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345a310-aa3b-41b3-b0aa-e127971f290f None admin] PATH_INFO: `/v3/groups/41e3de5fa88e4d14a42cbd73510dee66` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:45.406984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345a310-aa3b-41b3-b0aa-e127971f290f None admin] RBAC: Authorizing `identity:get_group(group_id=41e3de5fa88e4d14a42cbd73510dee66)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:45.408426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345a310-aa3b-41b3-b0aa-e127971f290f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:45.413936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 224/450] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:45 2022] GET /identity/v3/groups/41e3de5fa88e4d14a42cbd73510dee66 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:45.415881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345a310-aa3b-41b3-b0aa-e127971f290f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:45.470676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f8a72-5360-4bd8-b446-53304d7f46e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:45.472151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f8a72-5360-4bd8-b446-53304d7f46e0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:45.513949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345a310-aa3b-41b3-b0aa-e127971f290f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:45.524453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:45.550547 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367df9c8-5ead-4393-af9b-c292d628c7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:45.551207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367df9c8-5ead-4393-af9b-c292d628c7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:45.551490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367df9c8-5ead-4393-af9b-c292d628c7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:45.551768 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367df9c8-5ead-4393-af9b-c292d628c7b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:45.552599 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367df9c8-5ead-4393-af9b-c292d628c7b4 None admin] RBAC: Adding query filter params (group.id=41e3de5fa88e4d14a42cbd73510dee66, role.id=78a15fc7951d4f28beb57f54dace457d, scope.project.id=62e3ae8270a64809a9778530c2b3b7f0) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:45.552911 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367df9c8-5ead-4393-af9b-c292d628c7b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:45.554380 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367df9c8-5ead-4393-af9b-c292d628c7b4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:45.560432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 225/451] 10.222.0.79 () {56 vars in 1376 bytes} [Mon Jul 4 09:55:45 2022] GET /identity/v3/role_assignments?group.id=41e3de5fa88e4d14a42cbd73510dee66&scope.project.id=62e3ae8270a64809a9778530c2b3b7f0&role.id=78a15fc7951d4f28beb57f54dace457d => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:45.572629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 227/452] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:55:45.578721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367df9c8-5ead-4393-af9b-c292d628c7b4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:45.686502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367df9c8-5ead-4393-af9b-c292d628c7b4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:45.724132 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74888741-4cf5-426f-bdbc-a29e529e9e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:45.725004 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74888741-4cf5-426f-bdbc-a29e529e9e46 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:45.725286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74888741-4cf5-426f-bdbc-a29e529e9e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:45.725560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74888741-4cf5-426f-bdbc-a29e529e9e46 None admin] PATH_INFO: `/v3/projects/ca4e68f3b1bd4167b76136fe537c9f10` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:45.727098 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74888741-4cf5-426f-bdbc-a29e529e9e46 None admin] RBAC: Authorizing `identity:get_project(project_id=ca4e68f3b1bd4167b76136fe537c9f10)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:45.728559 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74888741-4cf5-426f-bdbc-a29e529e9e46 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:45.730704 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 226/453] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:55:45 2022] GET /identity/v3/projects/ca4e68f3b1bd4167b76136fe537c9f10 => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:45.736158 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f8a72-5360-4bd8-b446-53304d7f46e0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:45.772210 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:45.839245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f8a72-5360-4bd8-b446-53304d7f46e0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:45.873298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98bb7b-1f19-4a40-8c76-e8670741b3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:45.874184 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98bb7b-1f19-4a40-8c76-e8670741b3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:45.874458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98bb7b-1f19-4a40-8c76-e8670741b3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:45.874725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98bb7b-1f19-4a40-8c76-e8670741b3b2 None admin] PATH_INFO: `/v3/groups/6ef6458bfebf408bbfc4441465944997` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:45.875986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98bb7b-1f19-4a40-8c76-e8670741b3b2 None admin] RBAC: Authorizing `identity:get_group(group_id=6ef6458bfebf408bbfc4441465944997)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:45.877454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98bb7b-1f19-4a40-8c76-e8670741b3b2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:45.879105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 228/454] 10.222.0.79 () {56 vars in 1180 bytes} [Mon Jul 4 09:55:45 2022] GET /identity/v3/groups/6ef6458bfebf408bbfc4441465944997 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:45.884997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74888741-4cf5-426f-bdbc-a29e529e9e46 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:45.930353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:45.985950 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74888741-4cf5-426f-bdbc-a29e529e9e46 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:46.019462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8f6f5-2fc6-436e-9755-5fa8a79b0198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:46.020113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8f6f5-2fc6-436e-9755-5fa8a79b0198 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:46.020390 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8f6f5-2fc6-436e-9755-5fa8a79b0198 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:46.020665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8f6f5-2fc6-436e-9755-5fa8a79b0198 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:46.021490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8f6f5-2fc6-436e-9755-5fa8a79b0198 None admin] RBAC: Adding query filter params (group.id=6ef6458bfebf408bbfc4441465944997, role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=ca4e68f3b1bd4167b76136fe537c9f10) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:46.021828 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8f6f5-2fc6-436e-9755-5fa8a79b0198 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:46.023266 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8f6f5-2fc6-436e-9755-5fa8a79b0198 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:46.029097 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 227/455] 10.222.0.79 () {56 vars in 1376 bytes} [Mon Jul 4 09:55:45 2022] GET /identity/v3/role_assignments?group.id=6ef6458bfebf408bbfc4441465944997&scope.project.id=ca4e68f3b1bd4167b76136fe537c9f10&role.id=a3de9df253294d069dc1867fa26600c3 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:48.385099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fd015-38d4-47eb-a350-16506660c569 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:48.385640 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fd015-38d4-47eb-a350-16506660c569 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:48.385640 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fd015-38d4-47eb-a350-16506660c569 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:48.387182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 229/456] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:48.392856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208b408-e6dd-4f36-b5c4-b44f6c9867dd None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:48.393267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208b408-e6dd-4f36-b5c4-b44f6c9867dd None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:48.393458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208b408-e6dd-4f36-b5c4-b44f6c9867dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:48.418671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0208b408-e6dd-4f36-b5c4-b44f6c9867dd None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:48.419874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0208b408-e6dd-4f36-b5c4-b44f6c9867dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:48.509272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 228/457] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:48.527779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e370ba-467c-434d-843f-70e028368231 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:48.528404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e370ba-467c-434d-843f-70e028368231 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:48.528984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e370ba-467c-434d-843f-70e028368231 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:48.530843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 230/458] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:48.536509 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e0f68-9ae4-4ddf-b069-ff0e158d9a57 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:48.536799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e0f68-9ae4-4ddf-b069-ff0e158d9a57 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:48.537079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e0f68-9ae4-4ddf-b069-ff0e158d9a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:48.547304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e370ba-467c-434d-843f-70e028368231 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:48.562638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-446e0f68-9ae4-4ddf-b069-ff0e158d9a57 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:48.565412 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-446e0f68-9ae4-4ddf-b069-ff0e158d9a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:48.631360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:48.713846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 229/459] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:48.715284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e370ba-467c-434d-843f-70e028368231 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:48.717773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:48.740650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e0f68-9ae4-4ddf-b069-ff0e158d9a57 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:48.760298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f48bf-b449-401c-9e1e-42a15d59a09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:48.761381 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f48bf-b449-401c-9e1e-42a15d59a09e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:48.761827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f48bf-b449-401c-9e1e-42a15d59a09e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:48.762236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f48bf-b449-401c-9e1e-42a15d59a09e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:48.773099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f48bf-b449-401c-9e1e-42a15d59a09e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:48.774831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f48bf-b449-401c-9e1e-42a15d59a09e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-573f48bf-b449-401c-9e1e-42a15d59a09e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:48.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:48.793566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:48.794426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:48.794426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:48.794426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:48.794426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:48.796577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 231/460] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:48.802676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f48bf-b449-401c-9e1e-42a15d59a09e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:48.905216 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f48bf-b449-401c-9e1e-42a15d59a09e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:48.940978 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5940e2-02ad-4e4f-8d96-0ae096d9f490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:48.941969 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5940e2-02ad-4e4f-8d96-0ae096d9f490 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:48.942401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5940e2-02ad-4e4f-8d96-0ae096d9f490 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:48.942856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5940e2-02ad-4e4f-8d96-0ae096d9f490 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:48.943745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5940e2-02ad-4e4f-8d96-0ae096d9f490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:48.944215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5940e2-02ad-4e4f-8d96-0ae096d9f490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:48.945783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5940e2-02ad-4e4f-8d96-0ae096d9f490 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:48.964316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 232/461] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:48.970132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5940e2-02ad-4e4f-8d96-0ae096d9f490 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:48.981433 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e0f68-9ae4-4ddf-b069-ff0e158d9a57 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:48.989181 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:49.074822 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5940e2-02ad-4e4f-8d96-0ae096d9f490 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:49.077575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fcec8-c2c5-488d-8ad7-156571ded04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:49.078918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fcec8-c2c5-488d-8ad7-156571ded04d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:49.079443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fcec8-c2c5-488d-8ad7-156571ded04d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:49.079987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fcec8-c2c5-488d-8ad7-156571ded04d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:49.091752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fcec8-c2c5-488d-8ad7-156571ded04d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:49.093750 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fcec8-c2c5-488d-8ad7-156571ded04d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:49.111573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daf439-640a-4f79-8cc8-c37efa38f38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:49.112540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daf439-640a-4f79-8cc8-c37efa38f38f None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:49.112959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daf439-640a-4f79-8cc8-c37efa38f38f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:49.113366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daf439-640a-4f79-8cc8-c37efa38f38f None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:49.114236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daf439-640a-4f79-8cc8-c37efa38f38f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-859fcec8-c2c5-488d-8ad7-156571ded04d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:49.115238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:49.116217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:49.117185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:49.117185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:49.117185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:49.117185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:49.118137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 230/462] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:49.118503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daf439-640a-4f79-8cc8-c37efa38f38f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:49.119518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4faa7-444c-4f05-9622-9edefc0a6aaa None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:49.119871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4faa7-444c-4f05-9622-9edefc0a6aaa None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:49.120304 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4faa7-444c-4f05-9622-9edefc0a6aaa None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:49.121888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 231/463] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:49.125932 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4faa7-444c-4f05-9622-9edefc0a6aaa None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:49.147880 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 233/464] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:55:48 2022] POST /identity/v3/users => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:55:49.153700 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:49.154401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bba76-5465-4d6d-9fb5-b9edcc2b1daa None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:49.154830 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bba76-5465-4d6d-9fb5-b9edcc2b1daa None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:49.155240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bba76-5465-4d6d-9fb5-b9edcc2b1daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:49.180714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-dc7bba76-5465-4d6d-9fb5-b9edcc2b1daa None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:49.183856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-dc7bba76-5465-4d6d-9fb5-b9edcc2b1daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:49.282138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 234/465] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:49.283719 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03717969-d918-44b2-af9e-fcba28659716 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:49.284068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03717969-d918-44b2-af9e-fcba28659716 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:49.284349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03717969-d918-44b2-af9e-fcba28659716 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:49.289787 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 235/466] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:49.292045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4faa7-444c-4f05-9622-9edefc0a6aaa None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:49.297121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388467d-3eaa-41d8-85ae-bab57cc8216b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:49.297453 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388467d-3eaa-41d8-85ae-bab57cc8216b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:49.297764 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388467d-3eaa-41d8-85ae-bab57cc8216b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:49.316883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:49.330487 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c388467d-3eaa-41d8-85ae-bab57cc8216b None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:49.332062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c388467d-3eaa-41d8-85ae-bab57cc8216b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:49.341470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8b41a-15aa-4299-96a6-0ee5d842ce94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:49.341932 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8b41a-15aa-4299-96a6-0ee5d842ce94 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:49.342214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8b41a-15aa-4299-96a6-0ee5d842ce94 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:49.342487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8b41a-15aa-4299-96a6-0ee5d842ce94 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:49.343211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8b41a-15aa-4299-96a6-0ee5d842ce94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:49.343514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8b41a-15aa-4299-96a6-0ee5d842ce94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:49.345306 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8b41a-15aa-4299-96a6-0ee5d842ce94 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:49.358931 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 232/467] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:49.359837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8b41a-15aa-4299-96a6-0ee5d842ce94 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:49.424229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 236/468] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 04 09:55:49.425040 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388467d-3eaa-41d8-85ae-bab57cc8216b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:49.449634 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:49.476941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8b41a-15aa-4299-96a6-0ee5d842ce94 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:49.514473 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d4812-5ba7-4b4e-aac9-e29e5e770be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:49.515465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d4812-5ba7-4b4e-aac9-e29e5e770be2 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:49.515837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d4812-5ba7-4b4e-aac9-e29e5e770be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:49.516253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d4812-5ba7-4b4e-aac9-e29e5e770be2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:49.528774 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d4812-5ba7-4b4e-aac9-e29e5e770be2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:49.530486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d4812-5ba7-4b4e-aac9-e29e5e770be2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:49.548409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388467d-3eaa-41d8-85ae-bab57cc8216b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-eb2d4812-5ba7-4b4e-aac9-e29e5e770be2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:49.549249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:49.550290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:49.551353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:49.551353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:49.551353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:49.551353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:49.553066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 233/469] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:49.554381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d4812-5ba7-4b4e-aac9-e29e5e770be2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:49.589808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:49.595680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053edd9-75b9-4470-88fc-a689fb558d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:49.596876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053edd9-75b9-4470-88fc-a689fb558d73 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:49.597400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053edd9-75b9-4470-88fc-a689fb558d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:49.597859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053edd9-75b9-4470-88fc-a689fb558d73 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:49.598711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053edd9-75b9-4470-88fc-a689fb558d73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:49.602501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053edd9-75b9-4470-88fc-a689fb558d73 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:49.633432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 237/470] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:55:49 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:55:49.635241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053edd9-75b9-4470-88fc-a689fb558d73 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:49.695184 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d4812-5ba7-4b4e-aac9-e29e5e770be2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:49.733159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:49.742524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c87ec9-9933-44fc-b512-9e031b09a1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:49.743475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c87ec9-9933-44fc-b512-9e031b09a1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:49.743758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c87ec9-9933-44fc-b512-9e031b09a1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:49.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c87ec9-9933-44fc-b512-9e031b09a1f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:49.766843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c87ec9-9933-44fc-b512-9e031b09a1f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:49.770611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c87ec9-9933-44fc-b512-9e031b09a1f8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-99c87ec9-9933-44fc-b512-9e031b09a1f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:49.796800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:49.797823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:49.798933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:49.798933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:49.798933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:49.798933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:49.801362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 234/471] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:49.806978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c87ec9-9933-44fc-b512-9e031b09a1f8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:49.832085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053edd9-75b9-4470-88fc-a689fb558d73 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:49.877415 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cf58f-f131-44b6-b32e-4291109861cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:49.879027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cf58f-f131-44b6-b32e-4291109861cb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:49.879318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cf58f-f131-44b6-b32e-4291109861cb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:49.879590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cf58f-f131-44b6-b32e-4291109861cb None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:49.880320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cf58f-f131-44b6-b32e-4291109861cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:49.880624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cf58f-f131-44b6-b32e-4291109861cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:49.882039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cf58f-f131-44b6-b32e-4291109861cb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:49.898772 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 238/472] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:49.903871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cf58f-f131-44b6-b32e-4291109861cb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:49.971207 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:49.977920 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c87ec9-9933-44fc-b512-9e031b09a1f8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:50.027995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a414d-8948-460b-9e7d-21833498c704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:50.029711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a414d-8948-460b-9e7d-21833498c704 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:50.030199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a414d-8948-460b-9e7d-21833498c704 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:50.030923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a414d-8948-460b-9e7d-21833498c704 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:50.032710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a414d-8948-460b-9e7d-21833498c704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:50.033220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a414d-8948-460b-9e7d-21833498c704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:50.036854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a414d-8948-460b-9e7d-21833498c704 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:50.037439 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cf58f-f131-44b6-b32e-4291109861cb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:50.055176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 235/473] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:50.056903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87334cf4-fedb-4c3c-9a6c-aa25c14b5e5e None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:50.056903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87334cf4-fedb-4c3c-9a6c-aa25c14b5e5e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:50.057148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87334cf4-fedb-4c3c-9a6c-aa25c14b5e5e None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:50.058533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 236/474] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:50.063407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87334cf4-fedb-4c3c-9a6c-aa25c14b5e5e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:50.086448 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6bd8a-ad0b-45b1-b622-c69dbf95c785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:50.087127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6bd8a-ad0b-45b1-b622-c69dbf95c785 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:50.087414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6bd8a-ad0b-45b1-b622-c69dbf95c785 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:50.087692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6bd8a-ad0b-45b1-b622-c69dbf95c785 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:50.088407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6bd8a-ad0b-45b1-b622-c69dbf95c785 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:50.089949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6bd8a-ad0b-45b1-b622-c69dbf95c785 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:50.115865 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 239/475] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:55:49 2022] POST /identity/v3/users => generated 249 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:55:50.117489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38aeb5-4dbb-481d-adf5-4b8341172331 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:50.118032 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38aeb5-4dbb-481d-adf5-4b8341172331 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:50.118541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38aeb5-4dbb-481d-adf5-4b8341172331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:50.126661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:50.138197 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-df38aeb5-4dbb-481d-adf5-4b8341172331 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:50.139622 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-df38aeb5-4dbb-481d-adf5-4b8341172331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:50.218450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 240/476] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:50.243579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38aeb5-4dbb-481d-adf5-4b8341172331 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:50.247017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87334cf4-fedb-4c3c-9a6c-aa25c14b5e5e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:50.289647 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:50.312834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd30ca3-f53f-4430-ba9f-b1b261fc7da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:50.316388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd30ca3-f53f-4430-ba9f-b1b261fc7da6 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:50.316875 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd30ca3-f53f-4430-ba9f-b1b261fc7da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:50.317325 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd30ca3-f53f-4430-ba9f-b1b261fc7da6 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:50.318311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd30ca3-f53f-4430-ba9f-b1b261fc7da6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:50.320426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd30ca3-f53f-4430-ba9f-b1b261fc7da6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:50.349941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 237/477] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:55:50 2022] POST /identity/v3/users => generated 248 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:55:50.392118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38aeb5-4dbb-481d-adf5-4b8341172331 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:50.432052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbdf26-60be-43dd-8ea0-6b58b2005f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:50.433580 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbdf26-60be-43dd-8ea0-6b58b2005f53 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:50.434098 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbdf26-60be-43dd-8ea0-6b58b2005f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:50.434717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbdf26-60be-43dd-8ea0-6b58b2005f53 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:50.435767 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbdf26-60be-43dd-8ea0-6b58b2005f53 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:50.437487 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbdf26-60be-43dd-8ea0-6b58b2005f53 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:50.446342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.sql.core [None req-33bbdf26-60be-43dd-8ea0-6b58b2005f53 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 09:55:50.446342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 04 09:55:50.446342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [parameters: {'id': 'c68b45e713da461d96b91a2fcd7fe555', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 04 09:55:50.446342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80099) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-33bbdf26-60be-43dd-8ea0-6b58b2005f53 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result.read() Jul 04 09:55:50.464471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return ref.to_dict() Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:55:50.465419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application yield resource Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise self.value Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application session.commit() Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 09:55:50.466488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.session.flush() Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application compat.raise_( Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 09:55:50.467613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:55:50.468808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result.read() Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:55:50.476058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [parameters: {'id': 'c68b45e713da461d96b91a2fcd7fe555', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:50.478079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 04 09:55:50.480008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:50.484091 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 241/478] 10.222.0.79 () {60 vars in 1131 bytes} [Mon Jul 4 09:55:50 2022] POST /identity/v3/roles => generated 144 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:50.487458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd30ca3-f53f-4430-ba9f-b1b261fc7da6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:50.605360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:50.610359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd30ca3-f53f-4430-ba9f-b1b261fc7da6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:50.649869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4fd2e-0722-4726-9bb9-d610bf7b47af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:50.651081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4fd2e-0722-4726-9bb9-d610bf7b47af None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:50.651575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4fd2e-0722-4726-9bb9-d610bf7b47af None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:50.652049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4fd2e-0722-4726-9bb9-d610bf7b47af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:50.663841 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4fd2e-0722-4726-9bb9-d610bf7b47af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-e2e4fd2e-0722-4726-9bb9-d610bf7b47af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:50.672504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:50.673404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:50.674239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:50.674239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:50.674239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:50.674239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:50.674239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:50.674239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:50.674239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:50.674239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:50.674239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:50.674239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:50.674239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:55:50.674239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:50.680957 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 238/479] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:55:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:50.683025 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbdf26-60be-43dd-8ea0-6b58b2005f53 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:50.786428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:50.790451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbdf26-60be-43dd-8ea0-6b58b2005f53 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:50.828064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee76ae-2cc6-40b8-973e-ce6baad2b54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:50.828977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee76ae-2cc6-40b8-973e-ce6baad2b54a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:50.829299 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee76ae-2cc6-40b8-973e-ce6baad2b54a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:50.829588 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee76ae-2cc6-40b8-973e-ce6baad2b54a None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:50.830447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee76ae-2cc6-40b8-973e-ce6baad2b54a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:50.830757 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee76ae-2cc6-40b8-973e-ce6baad2b54a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:50.832166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee76ae-2cc6-40b8-973e-ce6baad2b54a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:50.845755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 242/480] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:51.545653 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee54ed-8933-4da2-bc19-5f895e7edf77 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:51.546174 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee54ed-8933-4da2-bc19-5f895e7edf77 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:51.546249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee54ed-8933-4da2-bc19-5f895e7edf77 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:51.548262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 239/481] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:51.553388 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32511e0-9f60-4828-baab-3d83605f4c1b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:51.553852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32511e0-9f60-4828-baab-3d83605f4c1b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:51.554023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32511e0-9f60-4828-baab-3d83605f4c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:51.578906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b32511e0-9f60-4828-baab-3d83605f4c1b None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:51.580398 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b32511e0-9f60-4828-baab-3d83605f4c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:51.676158 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 243/482] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:51.723129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee54ed-8933-4da2-bc19-5f895e7edf77 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:51.839995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee54ed-8933-4da2-bc19-5f895e7edf77 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:51.874684 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72267442-030c-43d6-bb04-079811340174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:51.875692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72267442-030c-43d6-bb04-079811340174 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:51.876202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72267442-030c-43d6-bb04-079811340174 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:51.876516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72267442-030c-43d6-bb04-079811340174 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:51.886993 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72267442-030c-43d6-bb04-079811340174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-72267442-030c-43d6-bb04-079811340174 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:51.895131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:51.896111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:51.896890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:51.896890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:51.896890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:51.896890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:51.896890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:51.896890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:51.896890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:51.896890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:51.896890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:51.896890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:51.896890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:51.896890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:51.896890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 240/483] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:55:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:51.902383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32511e0-9f60-4828-baab-3d83605f4c1b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:52.026867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:52.036428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32511e0-9f60-4828-baab-3d83605f4c1b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:52.071047 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be760c27-ec6e-446c-869d-d4e02b6940ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:52.071725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be760c27-ec6e-446c-869d-d4e02b6940ea None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:52.072006 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be760c27-ec6e-446c-869d-d4e02b6940ea None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:52.072284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be760c27-ec6e-446c-869d-d4e02b6940ea None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:52.073038 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be760c27-ec6e-446c-869d-d4e02b6940ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:52.073349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be760c27-ec6e-446c-869d-d4e02b6940ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:52.074774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be760c27-ec6e-446c-869d-d4e02b6940ea None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:52.083468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 244/484] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:55:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:52.088816 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72267442-030c-43d6-bb04-079811340174 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:52.134057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:52.218359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72267442-030c-43d6-bb04-079811340174 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:52.254702 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f8320-0371-482e-851e-cb71d9f37ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:52.255513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f8320-0371-482e-851e-cb71d9f37ceb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:52.255798 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f8320-0371-482e-851e-cb71d9f37ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:52.256072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f8320-0371-482e-851e-cb71d9f37ceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:52.267946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f8320-0371-482e-851e-cb71d9f37ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:52.269451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f8320-0371-482e-851e-cb71d9f37ceb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-0b2f8320-0371-482e-851e-cb71d9f37ceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:52.287589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:52.288508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:52.289961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:52.289961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:52.289961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:52.289961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:52.289961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 241/485] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:52.294913 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be760c27-ec6e-446c-869d-d4e02b6940ea None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:52.408136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269de75-6ef4-4e22-9ae4-138d677facea None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:52.408381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269de75-6ef4-4e22-9ae4-138d677facea None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:52.408616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269de75-6ef4-4e22-9ae4-138d677facea None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:52.414382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 242/486] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:52.420205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706a6fb-e31d-4d99-9d36-a52d69f73d15 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:52.420496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706a6fb-e31d-4d99-9d36-a52d69f73d15 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:52.420779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706a6fb-e31d-4d99-9d36-a52d69f73d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:52.452809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a706a6fb-e31d-4d99-9d36-a52d69f73d15 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:52.454090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a706a6fb-e31d-4d99-9d36-a52d69f73d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:52.458371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be760c27-ec6e-446c-869d-d4e02b6940ea None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:52.513125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0927456-e0d6-4129-994f-1256121398a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:52.513823 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0927456-e0d6-4129-994f-1256121398a6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:52.514115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0927456-e0d6-4129-994f-1256121398a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:52.514395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0927456-e0d6-4129-994f-1256121398a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:52.515127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0927456-e0d6-4129-994f-1256121398a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:52.515433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0927456-e0d6-4129-994f-1256121398a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:52.517048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0927456-e0d6-4129-994f-1256121398a6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:52.543840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 245/487] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:52.548797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0927456-e0d6-4129-994f-1256121398a6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:52.571022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:52.574210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:52.584380 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 243/488] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:52.606477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706a6fb-e31d-4d99-9d36-a52d69f73d15 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:52.731554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0927456-e0d6-4129-994f-1256121398a6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:52.766690 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706a6fb-e31d-4d99-9d36-a52d69f73d15 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:52.769005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ee902-90a5-4972-9cec-ed949f978456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:52.770080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ee902-90a5-4972-9cec-ed949f978456 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:52.770545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ee902-90a5-4972-9cec-ed949f978456 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:52.771068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ee902-90a5-4972-9cec-ed949f978456 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:52.778769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ee902-90a5-4972-9cec-ed949f978456 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:52.780448 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ee902-90a5-4972-9cec-ed949f978456 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-291ee902-90a5-4972-9cec-ed949f978456 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:55:52.796834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:52.798067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:52.799101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:52.799101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:52.799101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:55:52.799101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:55:52.799101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:55:52.799101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:55:52.799101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:55:52.799101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:55:52.799101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:55:52.799101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:52.801448 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 246/489] 10.222.0.79 () {56 vars in 1093 bytes} [Mon Jul 4 09:55:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:52.803046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162dee76-69d8-4415-aef0-2de7a3855cb8 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:52.803550 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162dee76-69d8-4415-aef0-2de7a3855cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:52.804028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162dee76-69d8-4415-aef0-2de7a3855cb8 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:52.805281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec1876-d3de-4595-8d49-1700f72936e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:52.805969 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec1876-d3de-4595-8d49-1700f72936e0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:52.806253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec1876-d3de-4595-8d49-1700f72936e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:52.811027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec1876-d3de-4595-8d49-1700f72936e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:52.811256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 247/490] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:52.812240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162dee76-69d8-4415-aef0-2de7a3855cb8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:52.822291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec1876-d3de-4595-8d49-1700f72936e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-a2ec1876-d3de-4595-8d49-1700f72936e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:52.832948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:52.834124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:52.835100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:52.835100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:52.835100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:52.835100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:52.835100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:52.835100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:52.835100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:52.835100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:52.835100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:52.835100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:52.835100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:52.835100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:52.836406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 244/491] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:55:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:52.838865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388ff2d-a5ad-4544-a103-2c7ea886a256 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:52.839338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388ff2d-a5ad-4544-a103-2c7ea886a256 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:52.839774 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388ff2d-a5ad-4544-a103-2c7ea886a256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:52.865754 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3388ff2d-a5ad-4544-a103-2c7ea886a256 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:52.867337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3388ff2d-a5ad-4544-a103-2c7ea886a256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:52.909043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:52.929673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:52.943459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 245/492] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:52.944712 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388ff2d-a5ad-4544-a103-2c7ea886a256 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:52.951273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162dee76-69d8-4415-aef0-2de7a3855cb8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:52.996539 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24760b4-7339-454c-a238-d78fbe4d0b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:52.997596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24760b4-7339-454c-a238-d78fbe4d0b42 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:52.998106 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24760b4-7339-454c-a238-d78fbe4d0b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:52.998657 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24760b4-7339-454c-a238-d78fbe4d0b42 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:52.999978 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24760b4-7339-454c-a238-d78fbe4d0b42 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:53.000553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24760b4-7339-454c-a238-d78fbe4d0b42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:53.002458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24760b4-7339-454c-a238-d78fbe4d0b42 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:53.046806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 248/493] 10.222.0.79 () {56 vars in 1133 bytes} [Mon Jul 4 09:55:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:53.048474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24760b4-7339-454c-a238-d78fbe4d0b42 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:53.060508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388ff2d-a5ad-4544-a103-2c7ea886a256 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:53.105345 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c50067-3ee7-4050-bb00-d5a6aefd43f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:53.106068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c50067-3ee7-4050-bb00-d5a6aefd43f0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:53.106647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c50067-3ee7-4050-bb00-d5a6aefd43f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:53.106933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c50067-3ee7-4050-bb00-d5a6aefd43f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:53.107686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c50067-3ee7-4050-bb00-d5a6aefd43f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:53.107992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c50067-3ee7-4050-bb00-d5a6aefd43f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:53.109963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c50067-3ee7-4050-bb00-d5a6aefd43f0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:53.120298 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 246/494] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:55:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:53.127438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c50067-3ee7-4050-bb00-d5a6aefd43f0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:53.175961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24760b4-7339-454c-a238-d78fbe4d0b42 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:53.200282 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:53.208739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:53.216217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed8bcf-d07f-42fb-88bc-3c21f21bc85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:53.217313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed8bcf-d07f-42fb-88bc-3c21f21bc85d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:53.217850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed8bcf-d07f-42fb-88bc-3c21f21bc85d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:53.218322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed8bcf-d07f-42fb-88bc-3c21f21bc85d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:53.232665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed8bcf-d07f-42fb-88bc-3c21f21bc85d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-89ed8bcf-d07f-42fb-88bc-3c21f21bc85d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:53.242704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:53.243607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:53.244831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:53.244831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:53.244831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:53.244831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:53.244831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:53.244831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:53.244831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:53.244831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:53.244831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:53.244831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:53.244831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:53.244831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:53.244831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 249/495] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:55:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:53.245713 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed8bcf-d07f-42fb-88bc-3c21f21bc85d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:53.262596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c50067-3ee7-4050-bb00-d5a6aefd43f0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:53.299672 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106b5c7-1486-406c-8577-518e1c4bd7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:53.300704 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106b5c7-1486-406c-8577-518e1c4bd7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:53.301214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106b5c7-1486-406c-8577-518e1c4bd7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:53.301720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106b5c7-1486-406c-8577-518e1c4bd7a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:53.367005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed8bcf-d07f-42fb-88bc-3c21f21bc85d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:53.388279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106b5c7-1486-406c-8577-518e1c4bd7a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:53.390233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106b5c7-1486-406c-8577-518e1c4bd7a6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:53.402474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0110872-e0c9-494b-a52c-93c85e3229aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:53.403276 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0110872-e0c9-494b-a52c-93c85e3229aa None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:53.403565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0110872-e0c9-494b-a52c-93c85e3229aa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:53.403843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0110872-e0c9-494b-a52c-93c85e3229aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:53.414975 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0110872-e0c9-494b-a52c-93c85e3229aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:53.416463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0110872-e0c9-494b-a52c-93c85e3229aa None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-f0110872-e0c9-494b-a52c-93c85e3229aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:53.434752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:53.435729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:53.436840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:53.436840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:53.436840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:53.436840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:53.436840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 250/496] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:53.438499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0110872-e0c9-494b-a52c-93c85e3229aa None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:53.450777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:53.484499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:53.500460 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 247/497] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:53.502164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2024-0e1d-49af-8752-f8cb76d520c1 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:53.502456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2024-0e1d-49af-8752-f8cb76d520c1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:53.502732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2024-0e1d-49af-8752-f8cb76d520c1 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:53.504040 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 248/498] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:53.504940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc2024-0e1d-49af-8752-f8cb76d520c1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:53.550481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0110872-e0c9-494b-a52c-93c85e3229aa None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:53.586178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9959f-a2c7-4cfd-8e44-ff0b1c948c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:53.586848 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9959f-a2c7-4cfd-8e44-ff0b1c948c15 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:53.587132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9959f-a2c7-4cfd-8e44-ff0b1c948c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:53.587408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9959f-a2c7-4cfd-8e44-ff0b1c948c15 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:53.588151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9959f-a2c7-4cfd-8e44-ff0b1c948c15 None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:53.588477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9959f-a2c7-4cfd-8e44-ff0b1c948c15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:53.589900 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9959f-a2c7-4cfd-8e44-ff0b1c948c15 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:53.597800 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 251/499] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:55:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:53.599211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d93e0-e3b9-4a60-80d4-7912c72b11c9 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:53.599565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d93e0-e3b9-4a60-80d4-7912c72b11c9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:53.599913 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d93e0-e3b9-4a60-80d4-7912c72b11c9 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:53.601543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 252/500] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:53.602272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d93e0-e3b9-4a60-80d4-7912c72b11c9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:53.612678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc2024-0e1d-49af-8752-f8cb76d520c1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:53.647517 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f2b8b-736f-4661-b294-dd48eb681d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:53.648192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f2b8b-736f-4661-b294-dd48eb681d70 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:53.648476 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f2b8b-736f-4661-b294-dd48eb681d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:53.648752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f2b8b-736f-4661-b294-dd48eb681d70 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:53.649499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f2b8b-736f-4661-b294-dd48eb681d70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:53.649823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f2b8b-736f-4661-b294-dd48eb681d70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:53.651213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f2b8b-736f-4661-b294-dd48eb681d70 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:53.662767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 249/501] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:53.663954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7fb5c-9e56-48bc-8ac4-43d54e55451c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:53.664247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7fb5c-9e56-48bc-8ac4-43d54e55451c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:53.664534 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7fb5c-9e56-48bc-8ac4-43d54e55451c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:53.680490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-40f7fb5c-9e56-48bc-8ac4-43d54e55451c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:53.681699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-40f7fb5c-9e56-48bc-8ac4-43d54e55451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:53.689722 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:53.705664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d93e0-e3b9-4a60-80d4-7912c72b11c9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:53.708084 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:53.741342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e1d44-41c7-401c-94e9-13a911fd0ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:53.742173 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e1d44-41c7-401c-94e9-13a911fd0ebd None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:53.742458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e1d44-41c7-401c-94e9-13a911fd0ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:53.742726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e1d44-41c7-401c-94e9-13a911fd0ebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:53.751578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 250/502] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:53.752420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7fb5c-9e56-48bc-8ac4-43d54e55451c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:53.761355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e1d44-41c7-401c-94e9-13a911fd0ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:53.763366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e1d44-41c7-401c-94e9-13a911fd0ebd None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d23e1d44-41c7-401c-94e9-13a911fd0ebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:53.784327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:53.785266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:53.786170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:53.786170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:53.786170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:53.786170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:53.786170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 253/503] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:53.786730 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad46d18-a46d-4f4e-a969-acc172e18a13 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:53.787011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad46d18-a46d-4f4e-a969-acc172e18a13 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:53.787285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad46d18-a46d-4f4e-a969-acc172e18a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:53.804196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-aad46d18-a46d-4f4e-a969-acc172e18a13 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:53.805328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-aad46d18-a46d-4f4e-a969-acc172e18a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:53.862556 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7fb5c-9e56-48bc-8ac4-43d54e55451c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:53.876371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 254/504] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:53.877235 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad46d18-a46d-4f4e-a969-acc172e18a13 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:53.899112 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ecdee-7b86-4231-b43d-6003f23a512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:53.899921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ecdee-7b86-4231-b43d-6003f23a512d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:53.900201 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ecdee-7b86-4231-b43d-6003f23a512d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:53.900472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ecdee-7b86-4231-b43d-6003f23a512d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:53.910934 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ecdee-7b86-4231-b43d-6003f23a512d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:53.912414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ecdee-7b86-4231-b43d-6003f23a512d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:53.914191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b25ecdee-7b86-4231-b43d-6003f23a512d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:53.930053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:53.931192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:53.936883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:53.936883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:53.936883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:53.936883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:53.936883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 251/505] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:53.936883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ecdee-7b86-4231-b43d-6003f23a512d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:53.966505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:53.982125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad46d18-a46d-4f4e-a969-acc172e18a13 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:54.014615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304f0e9-df41-438d-bb8b-e5d64b80c760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:54.015684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304f0e9-df41-438d-bb8b-e5d64b80c760 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:54.016088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304f0e9-df41-438d-bb8b-e5d64b80c760 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:54.016591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304f0e9-df41-438d-bb8b-e5d64b80c760 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:54.023515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304f0e9-df41-438d-bb8b-e5d64b80c760 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:54.025040 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304f0e9-df41-438d-bb8b-e5d64b80c760 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:54.031872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ecdee-7b86-4231-b43d-6003f23a512d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-a304f0e9-df41-438d-bb8b-e5d64b80c760 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:55:54.039385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:54.040356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:54.041432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:54.041432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:54.041432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:55:54.041432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:55:54.041432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:55:54.041432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:55:54.041432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:55:54.041432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:55:54.041432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:55:54.041432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:54.045127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 255/506] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:54.046012 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304f0e9-df41-438d-bb8b-e5d64b80c760 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:54.067842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65a281-0176-4ab4-95b1-991e197de7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:54.068569 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65a281-0176-4ab4-95b1-991e197de7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:54.068821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65a281-0176-4ab4-95b1-991e197de7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:54.069091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65a281-0176-4ab4-95b1-991e197de7d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:54.080427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65a281-0176-4ab4-95b1-991e197de7d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-4f65a281-0176-4ab4-95b1-991e197de7d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:54.088138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:54.089217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:54.090206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:54.090206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:54.090206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:54.090206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:54.090206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:54.090206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:54.090206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:54.090206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:54.090206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:54.090206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:54.090206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:54.090206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:54.090206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 252/507] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:55:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:54.092214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65a281-0176-4ab4-95b1-991e197de7d0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:54.148271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:54.154910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304f0e9-df41-438d-bb8b-e5d64b80c760 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:54.189980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a98579-2ca8-4500-b178-da37d64687d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:54.190651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a98579-2ca8-4500-b178-da37d64687d1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:54.190926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a98579-2ca8-4500-b178-da37d64687d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:54.191204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a98579-2ca8-4500-b178-da37d64687d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:54.191948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a98579-2ca8-4500-b178-da37d64687d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:54.192256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a98579-2ca8-4500-b178-da37d64687d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:54.193680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a98579-2ca8-4500-b178-da37d64687d1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:54.193983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65a281-0176-4ab4-95b1-991e197de7d0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:54.205245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 256/508] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:54.206086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a98579-2ca8-4500-b178-da37d64687d1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:54.218323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:54.230995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22348a-b310-421c-83ed-23125a341ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:54.231806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22348a-b310-421c-83ed-23125a341ccf None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:54.232118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22348a-b310-421c-83ed-23125a341ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:54.232394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22348a-b310-421c-83ed-23125a341ccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:54.243757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22348a-b310-421c-83ed-23125a341ccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:54.245230 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22348a-b310-421c-83ed-23125a341ccf None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-bc22348a-b310-421c-83ed-23125a341ccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:54.262603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:54.263440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:54.264310 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:54.264310 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:54.264310 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:54.264310 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:54.264310 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 253/509] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:54.264565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22348a-b310-421c-83ed-23125a341ccf None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:54.318192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a98579-2ca8-4500-b178-da37d64687d1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:54.351013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720990c1-54da-4dbd-a9d7-e3e8d6d52339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:54.351682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720990c1-54da-4dbd-a9d7-e3e8d6d52339 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:54.351966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720990c1-54da-4dbd-a9d7-e3e8d6d52339 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:54.352240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720990c1-54da-4dbd-a9d7-e3e8d6d52339 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:54.352961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720990c1-54da-4dbd-a9d7-e3e8d6d52339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:54.353262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720990c1-54da-4dbd-a9d7-e3e8d6d52339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:54.354702 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720990c1-54da-4dbd-a9d7-e3e8d6d52339 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:54.360721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22348a-b310-421c-83ed-23125a341ccf None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:54.366423 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 257/510] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:54.367213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720990c1-54da-4dbd-a9d7-e3e8d6d52339 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:54.367673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:54.397709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd47f79-e9b2-4c54-b07c-e0342dddd495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:54.398372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd47f79-e9b2-4c54-b07c-e0342dddd495 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:54.398652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd47f79-e9b2-4c54-b07c-e0342dddd495 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:54.398924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd47f79-e9b2-4c54-b07c-e0342dddd495 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:54.399757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd47f79-e9b2-4c54-b07c-e0342dddd495 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:54.400057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd47f79-e9b2-4c54-b07c-e0342dddd495 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:54.401500 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd47f79-e9b2-4c54-b07c-e0342dddd495 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:54.430808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:54.437245 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 254/511] 10.222.0.79 () {56 vars in 1137 bytes} [Mon Jul 4 09:55:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:54.438328 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd47f79-e9b2-4c54-b07c-e0342dddd495 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:54.469408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720990c1-54da-4dbd-a9d7-e3e8d6d52339 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:54.503691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2426633-b573-49c5-aaf1-8c925f1d75fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:54.504344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2426633-b573-49c5-aaf1-8c925f1d75fe None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:54.504628 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2426633-b573-49c5-aaf1-8c925f1d75fe None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:54.504907 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2426633-b573-49c5-aaf1-8c925f1d75fe None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:54.505684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2426633-b573-49c5-aaf1-8c925f1d75fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:54.505993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2426633-b573-49c5-aaf1-8c925f1d75fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:54.507389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2426633-b573-49c5-aaf1-8c925f1d75fe None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:54.515180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 258/512] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:55:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:54.516000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2426633-b573-49c5-aaf1-8c925f1d75fe None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:54.535705 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd47f79-e9b2-4c54-b07c-e0342dddd495 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:54.567947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59db49-28d5-4ad2-9170-0a7520254c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:54.568808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59db49-28d5-4ad2-9170-0a7520254c96 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:54.569038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59db49-28d5-4ad2-9170-0a7520254c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:54.569316 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59db49-28d5-4ad2-9170-0a7520254c96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:54.575870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59db49-28d5-4ad2-9170-0a7520254c96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:54.577344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59db49-28d5-4ad2-9170-0a7520254c96 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-0c59db49-28d5-4ad2-9170-0a7520254c96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:55:54.591749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:54.592637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:54.593505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:55:54.593505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:55:54.593505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:55:54.593505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:55:54.593505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:55:54.593505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:55:54.593505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:55:54.593505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:54.593505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 255/513] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:54.594530 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59db49-28d5-4ad2-9170-0a7520254c96 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:54.617675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2426633-b573-49c5-aaf1-8c925f1d75fe None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:54.621936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:54.643914 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:54.652004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab26a5f-a077-4b09-a191-182f560bd463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:54.652691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab26a5f-a077-4b09-a191-182f560bd463 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:54.652977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab26a5f-a077-4b09-a191-182f560bd463 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:54.653252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab26a5f-a077-4b09-a191-182f560bd463 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:54.654003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab26a5f-a077-4b09-a191-182f560bd463 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:54.654306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab26a5f-a077-4b09-a191-182f560bd463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:54.655692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab26a5f-a077-4b09-a191-182f560bd463 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:54.667368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 259/514] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:54.668153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab26a5f-a077-4b09-a191-182f560bd463 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:54.702650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59db49-28d5-4ad2-9170-0a7520254c96 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:54.735731 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e013-1c69-40ef-bdea-4632c52b1b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:54.736533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e013-1c69-40ef-bdea-4632c52b1b57 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:54.736817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e013-1c69-40ef-bdea-4632c52b1b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:54.737093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e013-1c69-40ef-bdea-4632c52b1b57 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:54.744401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727e013-1c69-40ef-bdea-4632c52b1b57 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:54.745889 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727e013-1c69-40ef-bdea-4632c52b1b57 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-0727e013-1c69-40ef-bdea-4632c52b1b57 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:55:54.761049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:54.762168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:54.763272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:54.763272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:54.763272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:55:54.763272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:55:54.763272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:55:54.763272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:55:54.763272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:55:54.763272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:55:54.763272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:55:54.763272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:54.763272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 256/515] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:55:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:54.763272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e013-1c69-40ef-bdea-4632c52b1b57 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:54.769246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab26a5f-a077-4b09-a191-182f560bd463 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:54.805075 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823ac50-bd9b-4a03-b821-3ae940eb284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:54.805974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823ac50-bd9b-4a03-b821-3ae940eb284d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:54.806319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823ac50-bd9b-4a03-b821-3ae940eb284d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:54.806625 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823ac50-bd9b-4a03-b821-3ae940eb284d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:54.852562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:54.861844 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:54.869988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e013-1c69-40ef-bdea-4632c52b1b57 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:54.874081 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823ac50-bd9b-4a03-b821-3ae940eb284d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:54.875750 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823ac50-bd9b-4a03-b821-3ae940eb284d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:54.904429 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74383f96-94cb-425e-badd-e826405f0010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:54.905226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74383f96-94cb-425e-badd-e826405f0010 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:54.905506 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74383f96-94cb-425e-badd-e826405f0010 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:54.905811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74383f96-94cb-425e-badd-e826405f0010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:54.916387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74383f96-94cb-425e-badd-e826405f0010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:54.917881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74383f96-94cb-425e-badd-e826405f0010 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-74383f96-94cb-425e-badd-e826405f0010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:54.934760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:54.935613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:54.936449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:54.936449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:54.936449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:54.936449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:54.936449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 257/516] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:54.936760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74383f96-94cb-425e-badd-e826405f0010 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:54.977108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 260/517] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:54.977956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823ac50-bd9b-4a03-b821-3ae940eb284d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:55.041139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74383f96-94cb-425e-badd-e826405f0010 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:55.064690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:55.073769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b0ef0-0c9e-4c06-aee1-d2705a49b447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:55.074433 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b0ef0-0c9e-4c06-aee1-d2705a49b447 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:55.074719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b0ef0-0c9e-4c06-aee1-d2705a49b447 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:55.074995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b0ef0-0c9e-4c06-aee1-d2705a49b447 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:55.075783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b0ef0-0c9e-4c06-aee1-d2705a49b447 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:55.076091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b0ef0-0c9e-4c06-aee1-d2705a49b447 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:55.077317 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823ac50-bd9b-4a03-b821-3ae940eb284d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:55.077547 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b0ef0-0c9e-4c06-aee1-d2705a49b447 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:55.081601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:55.095870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 258/518] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:55:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:55.096683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b0ef0-0c9e-4c06-aee1-d2705a49b447 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:55.113011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3013bef-d6bf-4232-8ba1-cad562bdbe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:55.113651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3013bef-d6bf-4232-8ba1-cad562bdbe41 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:55.113936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3013bef-d6bf-4232-8ba1-cad562bdbe41 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:55.114218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3013bef-d6bf-4232-8ba1-cad562bdbe41 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:55.114923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3013bef-d6bf-4232-8ba1-cad562bdbe41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:55.116395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3013bef-d6bf-4232-8ba1-cad562bdbe41 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:55.138990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 261/519] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:55:54 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:55:55.139812 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3013bef-d6bf-4232-8ba1-cad562bdbe41 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:55.213339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b0ef0-0c9e-4c06-aee1-d2705a49b447 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:55.254990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488157bf-1296-4206-aed3-b6d9737b6583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:55.256025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488157bf-1296-4206-aed3-b6d9737b6583 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:55.256315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488157bf-1296-4206-aed3-b6d9737b6583 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:55.256621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488157bf-1296-4206-aed3-b6d9737b6583 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:55.257482 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488157bf-1296-4206-aed3-b6d9737b6583 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:55.257846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488157bf-1296-4206-aed3-b6d9737b6583 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:55.258367 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3013bef-d6bf-4232-8ba1-cad562bdbe41 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:55.259346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488157bf-1296-4206-aed3-b6d9737b6583 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:55.281588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 259/520] 10.222.0.79 () {56 vars in 1139 bytes} [Mon Jul 4 09:55:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:55.282503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488157bf-1296-4206-aed3-b6d9737b6583 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:55.304833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e917c-446e-4015-92cd-6783e6d42be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:55.305506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e917c-446e-4015-92cd-6783e6d42be6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:55.305876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e917c-446e-4015-92cd-6783e6d42be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:55.306088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e917c-446e-4015-92cd-6783e6d42be6 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:55.306838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e917c-446e-4015-92cd-6783e6d42be6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:55.307729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e917c-446e-4015-92cd-6783e6d42be6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:55.308628 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e917c-446e-4015-92cd-6783e6d42be6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:55.328039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 262/521] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:55.329104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e917c-446e-4015-92cd-6783e6d42be6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:55.334956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:55.355286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:55.405939 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488157bf-1296-4206-aed3-b6d9737b6583 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:55.435726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e917c-446e-4015-92cd-6783e6d42be6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:55.443837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b33314-5432-43bc-9e54-0125fb17e796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:55.444508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b33314-5432-43bc-9e54-0125fb17e796 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:55.444793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b33314-5432-43bc-9e54-0125fb17e796 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:55.445064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b33314-5432-43bc-9e54-0125fb17e796 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:55.455672 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b33314-5432-43bc-9e54-0125fb17e796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:55.457149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b33314-5432-43bc-9e54-0125fb17e796 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:55.471496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e933ad-eaa4-4c99-8839-d939746576fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:55.472155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e933ad-eaa4-4c99-8839-d939746576fd None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:55.472432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e933ad-eaa4-4c99-8839-d939746576fd None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:55.472699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e933ad-eaa4-4c99-8839-d939746576fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:55.473513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e933ad-eaa4-4c99-8839-d939746576fd None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=ffc2409cb7fc4373a74ddfb48d501d30) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:55.473842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e933ad-eaa4-4c99-8839-d939746576fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-39b33314-5432-43bc-9e54-0125fb17e796 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:55.474862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:55.476003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:55.476859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:55.476859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:55.476859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:55.476859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:55.477066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e933ad-eaa4-4c99-8839-d939746576fd None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:55.480977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 260/522] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:55.489981 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 263/523] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:55 2022] GET /identity/v3/role_assignments?user.id=ffc2409cb7fc4373a74ddfb48d501d30&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:55.490227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b33314-5432-43bc-9e54-0125fb17e796 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:55.490324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e933ad-eaa4-4c99-8839-d939746576fd None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:55.569778 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:55.620691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e933ad-eaa4-4c99-8839-d939746576fd None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:55.641603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b33314-5432-43bc-9e54-0125fb17e796 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:55.650974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:55.684498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76454b35-d97a-4018-ba9f-cc143ee0f967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:55.685266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76454b35-d97a-4018-ba9f-cc143ee0f967 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:55.685548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76454b35-d97a-4018-ba9f-cc143ee0f967 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:55.685853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76454b35-d97a-4018-ba9f-cc143ee0f967 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:55.691458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d5e97-72e2-4f8f-b1bf-e72286222784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:55.692121 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d5e97-72e2-4f8f-b1bf-e72286222784 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:55.692410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d5e97-72e2-4f8f-b1bf-e72286222784 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:55.692688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d5e97-72e2-4f8f-b1bf-e72286222784 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:55.694509 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76454b35-d97a-4018-ba9f-cc143ee0f967 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:55.695993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76454b35-d97a-4018-ba9f-cc143ee0f967 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-76454b35-d97a-4018-ba9f-cc143ee0f967 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:55:55.711885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:55.712976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:55.713902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:55.713902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:55.713902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:55:55.713902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:55:55.713902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:55:55.713902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:55:55.713902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:55:55.713902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:55:55.713902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:55:55.713902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:55.713902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 264/524] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:55:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:55.714466 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76454b35-d97a-4018-ba9f-cc143ee0f967 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:55.770123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d5e97-72e2-4f8f-b1bf-e72286222784 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:55.771610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d5e97-72e2-4f8f-b1bf-e72286222784 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:55.833999 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76454b35-d97a-4018-ba9f-cc143ee0f967 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:55.866629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:55.870448 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85ba0f-8418-45b0-a5ce-39de3de3e17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:55.871122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85ba0f-8418-45b0-a5ce-39de3de3e17a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:55.871458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85ba0f-8418-45b0-a5ce-39de3de3e17a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:55.871742 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85ba0f-8418-45b0-a5ce-39de3de3e17a None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:55.872675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85ba0f-8418-45b0-a5ce-39de3de3e17a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:55.872999 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85ba0f-8418-45b0-a5ce-39de3de3e17a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:55.874459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85ba0f-8418-45b0-a5ce-39de3de3e17a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:55.884423 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 261/525] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:55.885354 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d5e97-72e2-4f8f-b1bf-e72286222784 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:55.891928 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 265/526] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:55.892825 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85ba0f-8418-45b0-a5ce-39de3de3e17a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:55.894312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:55.992022 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d5e97-72e2-4f8f-b1bf-e72286222784 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:56.000225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85ba0f-8418-45b0-a5ce-39de3de3e17a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:56.029023 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd118aad-4fd5-4bc7-8350-7aac4a47ea51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:56.029834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd118aad-4fd5-4bc7-8350-7aac4a47ea51 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:56.030117 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd118aad-4fd5-4bc7-8350-7aac4a47ea51 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:56.030383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd118aad-4fd5-4bc7-8350-7aac4a47ea51 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:56.031224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd118aad-4fd5-4bc7-8350-7aac4a47ea51 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:56.031849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd118aad-4fd5-4bc7-8350-7aac4a47ea51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:56.033387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd118aad-4fd5-4bc7-8350-7aac4a47ea51 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:56.041190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d113ad3-e7aa-4004-a47d-02ae82351691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:56.042037 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d113ad3-e7aa-4004-a47d-02ae82351691 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:56.042425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d113ad3-e7aa-4004-a47d-02ae82351691 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:56.042710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d113ad3-e7aa-4004-a47d-02ae82351691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:56.053725 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 262/527] 10.222.0.79 () {56 vars in 1137 bytes} [Mon Jul 4 09:55:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:56.054140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d113ad3-e7aa-4004-a47d-02ae82351691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:56.054576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd118aad-4fd5-4bc7-8350-7aac4a47ea51 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:56.055739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d113ad3-e7aa-4004-a47d-02ae82351691 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1d113ad3-e7aa-4004-a47d-02ae82351691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:56.074773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:56.075689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:56.076526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:56.076526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:56.076526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:56.076526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:56.076526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 266/528] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:56.077229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d113ad3-e7aa-4004-a47d-02ae82351691 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:56.127282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:56.128669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:56.177060 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd118aad-4fd5-4bc7-8350-7aac4a47ea51 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:56.192053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d113ad3-e7aa-4004-a47d-02ae82351691 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:56.213882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7de09-c9e5-4412-9773-066096d4a81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:56.214933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7de09-c9e5-4412-9773-066096d4a81c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:56.215450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7de09-c9e5-4412-9773-066096d4a81c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:56.215955 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7de09-c9e5-4412-9773-066096d4a81c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:56.223806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7de09-c9e5-4412-9773-066096d4a81c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:56.225447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7de09-c9e5-4412-9773-066096d4a81c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:56.228921 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db7a80-d64b-4034-9045-35913caeb933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:56.229589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db7a80-d64b-4034-9045-35913caeb933 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:56.229903 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db7a80-d64b-4034-9045-35913caeb933 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:56.230184 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db7a80-d64b-4034-9045-35913caeb933 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b5c7de09-c9e5-4412-9773-066096d4a81c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:55:56.241470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:56.242624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:56.243688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:55:56.243688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:55:56.243688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:55:56.243688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:55:56.243688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:55:56.243688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:55:56.243688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:55:56.243688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:56.245821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 263/529] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:56.246859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7de09-c9e5-4412-9773-066096d4a81c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:56.310755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db7a80-d64b-4034-9045-35913caeb933 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:56.312388 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db7a80-d64b-4034-9045-35913caeb933 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:56.357636 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:56.382601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7de09-c9e5-4412-9773-066096d4a81c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:56.415457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:56.421040 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 267/530] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:56.422484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db7a80-d64b-4034-9045-35913caeb933 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:56.430267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ae91d-4b48-4354-ac06-f4b7c35a2771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:56.431148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ae91d-4b48-4354-ac06-f4b7c35a2771 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:56.432073 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ae91d-4b48-4354-ac06-f4b7c35a2771 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:56.432508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ae91d-4b48-4354-ac06-f4b7c35a2771 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:56.433373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ae91d-4b48-4354-ac06-f4b7c35a2771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:56.434304 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ae91d-4b48-4354-ac06-f4b7c35a2771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:56.436262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ae91d-4b48-4354-ac06-f4b7c35a2771 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:56.456499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 264/531] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:56.456499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ae91d-4b48-4354-ac06-f4b7c35a2771 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:56.537779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db7a80-d64b-4034-9045-35913caeb933 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:56.573144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4311c0-d752-441d-a5a9-619e0985838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:56.574067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4311c0-d752-441d-a5a9-619e0985838a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:56.574490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4311c0-d752-441d-a5a9-619e0985838a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:56.574931 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4311c0-d752-441d-a5a9-619e0985838a None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:56.575857 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4311c0-d752-441d-a5a9-619e0985838a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:56.576306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4311c0-d752-441d-a5a9-619e0985838a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:56.578020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4311c0-d752-441d-a5a9-619e0985838a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:56.586134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ae91d-4b48-4354-ac06-f4b7c35a2771 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:56.599329 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:56.609759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 268/532] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:55:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:56.610703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4311c0-d752-441d-a5a9-619e0985838a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:56.624448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290a2ab-6209-4515-8aa8-0c83f429a5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:56.625594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290a2ab-6209-4515-8aa8-0c83f429a5bc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:56.626109 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290a2ab-6209-4515-8aa8-0c83f429a5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:56.626603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290a2ab-6209-4515-8aa8-0c83f429a5bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:56.637341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290a2ab-6209-4515-8aa8-0c83f429a5bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:56.639048 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290a2ab-6209-4515-8aa8-0c83f429a5bc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-8290a2ab-6209-4515-8aa8-0c83f429a5bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:56.657179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:56.658699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:56.659536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:56.659536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:56.659536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:56.659536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:56.661844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 265/533] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:56.662952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290a2ab-6209-4515-8aa8-0c83f429a5bc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:56.701396 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:56.730853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4311c0-d752-441d-a5a9-619e0985838a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:56.781513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290a2ab-6209-4515-8aa8-0c83f429a5bc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:56.783174 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321cbb5-b193-46c9-8ebb-4f785520fedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:56.784095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321cbb5-b193-46c9-8ebb-4f785520fedf None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:56.784381 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321cbb5-b193-46c9-8ebb-4f785520fedf None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:56.784657 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321cbb5-b193-46c9-8ebb-4f785520fedf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:56.791937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321cbb5-b193-46c9-8ebb-4f785520fedf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:56.793456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321cbb5-b193-46c9-8ebb-4f785520fedf None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d321cbb5-b193-46c9-8ebb-4f785520fedf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:55:56.809499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:56.810396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:56.811219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:55:56.811219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:55:56.811219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:55:56.811219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:55:56.811219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:55:56.811219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:55:56.811219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:55:56.811219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:56.816261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 269/534] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:56.817325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321cbb5-b193-46c9-8ebb-4f785520fedf None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:56.823061 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061af14-04c4-4feb-af8d-23692fd51d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:56.824098 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061af14-04c4-4feb-af8d-23692fd51d40 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:56.824576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061af14-04c4-4feb-af8d-23692fd51d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:56.825088 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061af14-04c4-4feb-af8d-23692fd51d40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:56.826207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061af14-04c4-4feb-af8d-23692fd51d40 None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=fef685a17c804e2bbfea1e07cfdf5cfe) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:56.826762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061af14-04c4-4feb-af8d-23692fd51d40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:56.828440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061af14-04c4-4feb-af8d-23692fd51d40 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:56.834959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 266/535] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:56 2022] GET /identity/v3/role_assignments?user.id=fef685a17c804e2bbfea1e07cfdf5cfe&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:56.835972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061af14-04c4-4feb-af8d-23692fd51d40 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:56.910033 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:56.943204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061af14-04c4-4feb-af8d-23692fd51d40 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:56.947335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:56.985009 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321cbb5-b193-46c9-8ebb-4f785520fedf None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:57.010800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73164190-0985-4c80-9a8b-5ca9c9d506bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:57.011887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73164190-0985-4c80-9a8b-5ca9c9d506bd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:57.012387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73164190-0985-4c80-9a8b-5ca9c9d506bd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:57.018101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73164190-0985-4c80-9a8b-5ca9c9d506bd None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:57.018976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73164190-0985-4c80-9a8b-5ca9c9d506bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:57.019301 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73164190-0985-4c80-9a8b-5ca9c9d506bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:57.020843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73164190-0985-4c80-9a8b-5ca9c9d506bd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:57.038599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb909711-2c42-46ee-9ee4-78eac93343af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:57.039798 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb909711-2c42-46ee-9ee4-78eac93343af None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:57.040337 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb909711-2c42-46ee-9ee4-78eac93343af None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:57.040862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb909711-2c42-46ee-9ee4-78eac93343af None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:57.041859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb909711-2c42-46ee-9ee4-78eac93343af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:57.042433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb909711-2c42-46ee-9ee4-78eac93343af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:57.044765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb909711-2c42-46ee-9ee4-78eac93343af None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:57.058130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 267/536] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:55:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:57.067166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73164190-0985-4c80-9a8b-5ca9c9d506bd None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:57.071117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 270/537] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:57.078003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb909711-2c42-46ee-9ee4-78eac93343af None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:57.231909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73164190-0985-4c80-9a8b-5ca9c9d506bd None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:57.236956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb909711-2c42-46ee-9ee4-78eac93343af None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:57.264467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:57.290662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862dbac-9003-4a3e-a5eb-16c6c7820f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:57.291476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862dbac-9003-4a3e-a5eb-16c6c7820f0c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:57.291755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862dbac-9003-4a3e-a5eb-16c6c7820f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:57.292028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862dbac-9003-4a3e-a5eb-16c6c7820f0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:57.300089 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862dbac-9003-4a3e-a5eb-16c6c7820f0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:57.301506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862dbac-9003-4a3e-a5eb-16c6c7820f0c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-6862dbac-9003-4a3e-a5eb-16c6c7820f0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:55:57.319911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:57.320802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:57.321672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:55:57.321672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:55:57.321672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:55:57.321672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:55:57.321672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:55:57.321672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:55:57.321672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:55:57.321672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:57.322082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117ef4d-8af5-4a72-911b-fd96acdf7e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:57.322082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117ef4d-8af5-4a72-911b-fd96acdf7e04 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:57.322082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117ef4d-8af5-4a72-911b-fd96acdf7e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:57.322082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117ef4d-8af5-4a72-911b-fd96acdf7e04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:57.322856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 271/538] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:55:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:57.322960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117ef4d-8af5-4a72-911b-fd96acdf7e04 None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=3a1f11a4b8554a98b7626a82f01888c7) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:57.322960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117ef4d-8af5-4a72-911b-fd96acdf7e04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:57.324328 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117ef4d-8af5-4a72-911b-fd96acdf7e04 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:57.326692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862dbac-9003-4a3e-a5eb-16c6c7820f0c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:57.344012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 268/539] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:57 2022] GET /identity/v3/role_assignments?user.id=3a1f11a4b8554a98b7626a82f01888c7&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:57.344432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c07e5c-9434-4d1c-a5ee-08f7e3c54cd9 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:57.345043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c07e5c-9434-4d1c-a5ee-08f7e3c54cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:57.345462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c07e5c-9434-4d1c-a5ee-08f7e3c54cd9 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:57.347244 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 269/540] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:57.352765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1abf2-275a-4adb-8795-c9e867ba7b08 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:57.353388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1abf2-275a-4adb-8795-c9e867ba7b08 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:57.353830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1abf2-275a-4adb-8795-c9e867ba7b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:57.381962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5bf1abf2-275a-4adb-8795-c9e867ba7b08 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:57.383732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5bf1abf2-275a-4adb-8795-c9e867ba7b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:57.423854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:57.502248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862dbac-9003-4a3e-a5eb-16c6c7820f0c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:57.522710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 270/541] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:57.558570 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5d20f-d6d8-4a8f-91c2-fe8d7f4fcabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:57.559159 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5d20f-d6d8-4a8f-91c2-fe8d7f4fcabb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:57.559435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5d20f-d6d8-4a8f-91c2-fe8d7f4fcabb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:57.559704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5d20f-d6d8-4a8f-91c2-fe8d7f4fcabb None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:57.560488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5d20f-d6d8-4a8f-91c2-fe8d7f4fcabb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:57.560791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5d20f-d6d8-4a8f-91c2-fe8d7f4fcabb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:57.562353 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5d20f-d6d8-4a8f-91c2-fe8d7f4fcabb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:57.564035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1abf2-275a-4adb-8795-c9e867ba7b08 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:57.578602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 272/542] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:55:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:57.585497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5d20f-d6d8-4a8f-91c2-fe8d7f4fcabb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:57.638297 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:57.712678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5d20f-d6d8-4a8f-91c2-fe8d7f4fcabb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:57.755333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1abf2-275a-4adb-8795-c9e867ba7b08 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:57.758278 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27c370-3e84-4962-a885-abf73c4f6184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:57.758940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27c370-3e84-4962-a885-abf73c4f6184 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:57.759221 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27c370-3e84-4962-a885-abf73c4f6184 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:57.759500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27c370-3e84-4962-a885-abf73c4f6184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:57.760333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27c370-3e84-4962-a885-abf73c4f6184 None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=c5a4af83902d4370a832ed3ea3861b56) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:57.760648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27c370-3e84-4962-a885-abf73c4f6184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:57.762226 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27c370-3e84-4962-a885-abf73c4f6184 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:57.768832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 273/543] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:55:57 2022] GET /identity/v3/role_assignments?user.id=c5a4af83902d4370a832ed3ea3861b56&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:57.770222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ed290-a123-40a9-afbe-e0a4642c78a1 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:57.770509 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ed290-a123-40a9-afbe-e0a4642c78a1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:57.770789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ed290-a123-40a9-afbe-e0a4642c78a1 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:57.776477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 274/544] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:57.783630 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c930d1c-7fdc-483f-96dc-efb9db5ef7d1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:57.783630 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c930d1c-7fdc-483f-96dc-efb9db5ef7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:57.783630 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c930d1c-7fdc-483f-96dc-efb9db5ef7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:57.807674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515ce19-c526-4219-928f-fce521a13e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:57.809038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515ce19-c526-4219-928f-fce521a13e33 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:57.809474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515ce19-c526-4219-928f-fce521a13e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:57.809923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515ce19-c526-4219-928f-fce521a13e33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:57.818162 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5c930d1c-7fdc-483f-96dc-efb9db5ef7d1 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:57.819405 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5c930d1c-7fdc-483f-96dc-efb9db5ef7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:57.824593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515ce19-c526-4219-928f-fce521a13e33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-1515ce19-c526-4219-928f-fce521a13e33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:57.834212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:57.835381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:57.836386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:57.836386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:57.836386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:57.836386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:57.836386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:57.836386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:57.836386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:57.836386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:57.836386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:57.836386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:57.836386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:57.836386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:57.839296 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 271/545] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:55:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:57.851087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515ce19-c526-4219-928f-fce521a13e33 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:57.885129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:57.951248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 275/546] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:57.991841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c930d1c-7fdc-483f-96dc-efb9db5ef7d1 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:58.016991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515ce19-c526-4219-928f-fce521a13e33 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:58.054463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:58.072200 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24756a-5cda-402f-862a-d65e9d8f7bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:58.072804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24756a-5cda-402f-862a-d65e9d8f7bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:58.073090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24756a-5cda-402f-862a-d65e9d8f7bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:58.073372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24756a-5cda-402f-862a-d65e9d8f7bf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:58.074163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24756a-5cda-402f-862a-d65e9d8f7bf0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:58.074476 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24756a-5cda-402f-862a-d65e9d8f7bf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:58.075900 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24756a-5cda-402f-862a-d65e9d8f7bf0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:58.093021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 272/547] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:55:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:58.099185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24756a-5cda-402f-862a-d65e9d8f7bf0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:58.143157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c930d1c-7fdc-483f-96dc-efb9db5ef7d1 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:58.202312 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86e4f6-2aae-4cae-8205-e6382ad7a7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:58.203142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86e4f6-2aae-4cae-8205-e6382ad7a7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:58.203454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86e4f6-2aae-4cae-8205-e6382ad7a7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:58.203726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86e4f6-2aae-4cae-8205-e6382ad7a7e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:58.214567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86e4f6-2aae-4cae-8205-e6382ad7a7e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:58.221959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-fa86e4f6-2aae-4cae-8205-e6382ad7a7e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:58.222989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:58.223881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:58.225077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:58.225077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:58.225077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:58.225077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:58.225077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:58.225077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:58.225077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:58.225077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:58.225077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:58.225077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:58.225077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:58.225077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:58.225077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 276/548] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:55:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:58.231138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86e4f6-2aae-4cae-8205-e6382ad7a7e5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:58.334242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24756a-5cda-402f-862a-d65e9d8f7bf0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:58.363280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:58.374962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8645f8-77c9-4642-8754-36a00d887edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:58.376346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8645f8-77c9-4642-8754-36a00d887edc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:58.376631 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8645f8-77c9-4642-8754-36a00d887edc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:58.376910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8645f8-77c9-4642-8754-36a00d887edc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:58.379293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86e4f6-2aae-4cae-8205-e6382ad7a7e5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:58.400091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8645f8-77c9-4642-8754-36a00d887edc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:58.401564 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8645f8-77c9-4642-8754-36a00d887edc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-0c8645f8-77c9-4642-8754-36a00d887edc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:58.428784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:58.430205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:58.431217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:58.431217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:58.431217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:58.431217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:58.438181 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 273/549] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:58.444267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8645f8-77c9-4642-8754-36a00d887edc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:58.450073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763b43e-ec09-46e9-909f-d536488c709e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:58.450752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763b43e-ec09-46e9-909f-d536488c709e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:58.451037 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763b43e-ec09-46e9-909f-d536488c709e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:58.451315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763b43e-ec09-46e9-909f-d536488c709e None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:58.452068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763b43e-ec09-46e9-909f-d536488c709e None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:58.452381 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763b43e-ec09-46e9-909f-d536488c709e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:58.454211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763b43e-ec09-46e9-909f-d536488c709e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:58.469332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 277/550] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:55:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:58.475319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763b43e-ec09-46e9-909f-d536488c709e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:58.601007 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8645f8-77c9-4642-8754-36a00d887edc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:58.629421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:58.643525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763b43e-ec09-46e9-909f-d536488c709e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:58.645667 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9b019-8270-4b00-8dbc-37af16640601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:58.646472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9b019-8270-4b00-8dbc-37af16640601 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:58.646766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9b019-8270-4b00-8dbc-37af16640601 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:58.647048 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9b019-8270-4b00-8dbc-37af16640601 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:58.647796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9b019-8270-4b00-8dbc-37af16640601 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:58.648107 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9b019-8270-4b00-8dbc-37af16640601 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:58.649521 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9b019-8270-4b00-8dbc-37af16640601 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:58.671835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 274/551] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:58.679297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9b019-8270-4b00-8dbc-37af16640601 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:58.684246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2d141-abec-4df9-8325-54c5e354b967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:58.685275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2d141-abec-4df9-8325-54c5e354b967 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:58.686693 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2d141-abec-4df9-8325-54c5e354b967 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:58.687154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2d141-abec-4df9-8325-54c5e354b967 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:58.711234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2d141-abec-4df9-8325-54c5e354b967 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:58.714096 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2d141-abec-4df9-8325-54c5e354b967 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-6ef2d141-abec-4df9-8325-54c5e354b967 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:58.745485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:58.746552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:58.747512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:58.747512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:58.747512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:58.747512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:58.763747 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 278/552] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:58.770122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2d141-abec-4df9-8325-54c5e354b967 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:58.798372 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:58.886366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2d141-abec-4df9-8325-54c5e354b967 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:58.925109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc85b44-286c-4aed-8587-a108564384ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:58.925826 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc85b44-286c-4aed-8587-a108564384ec None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:58.926112 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc85b44-286c-4aed-8587-a108564384ec None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:58.926816 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc85b44-286c-4aed-8587-a108564384ec None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:58.927742 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc85b44-286c-4aed-8587-a108564384ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:58.928103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc85b44-286c-4aed-8587-a108564384ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:58.929521 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc85b44-286c-4aed-8587-a108564384ec None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:58.931750 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9b019-8270-4b00-8dbc-37af16640601 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:58.943808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 279/553] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:58.951089 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc85b44-286c-4aed-8587-a108564384ec None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:58.998216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64893e62-66e1-4c27-9c47-e72e571875cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:58.999063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64893e62-66e1-4c27-9c47-e72e571875cf None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:58.999395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64893e62-66e1-4c27-9c47-e72e571875cf None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:58.999673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64893e62-66e1-4c27-9c47-e72e571875cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:59.008035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64893e62-66e1-4c27-9c47-e72e571875cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:59.009512 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64893e62-66e1-4c27-9c47-e72e571875cf None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-64893e62-66e1-4c27-9c47-e72e571875cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:55:59.026185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:59.027159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:59.028153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:59.028153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:59.028153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:55:59.028153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:55:59.028153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:55:59.028153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:55:59.028153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:55:59.028153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:55:59.028153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:55:59.028153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:59.028668 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 275/554] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:55:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:55:59.038785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64893e62-66e1-4c27-9c47-e72e571875cf None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:59.074240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:59.124442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc85b44-286c-4aed-8587-a108564384ec None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:59.146497 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:59.188715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa97ae9-07e2-4812-89c1-552635cb0871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:59.189527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa97ae9-07e2-4812-89c1-552635cb0871 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:59.189831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa97ae9-07e2-4812-89c1-552635cb0871 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:59.190107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa97ae9-07e2-4812-89c1-552635cb0871 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:59.204847 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa97ae9-07e2-4812-89c1-552635cb0871 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:59.206397 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa97ae9-07e2-4812-89c1-552635cb0871 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-9aa97ae9-07e2-4812-89c1-552635cb0871 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:55:59.232522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:59.234961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:59.237859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:59.237859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:59.237859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:55:59.237859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:55:59.237859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:55:59.237859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:55:59.237859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:55:59.237859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:55:59.237859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:55:59.237859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:59.242183 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64893e62-66e1-4c27-9c47-e72e571875cf None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:59.244019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 280/555] 10.222.0.79 () {56 vars in 1093 bytes} [Mon Jul 4 09:55:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:59.250330 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa97ae9-07e2-4812-89c1-552635cb0871 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:59.319021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e80ea6-1a18-46a3-86b7-a641319ed702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:59.319765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e80ea6-1a18-46a3-86b7-a641319ed702 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:59.320079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e80ea6-1a18-46a3-86b7-a641319ed702 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:59.320362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e80ea6-1a18-46a3-86b7-a641319ed702 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:59.321213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e80ea6-1a18-46a3-86b7-a641319ed702 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:59.321522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e80ea6-1a18-46a3-86b7-a641319ed702 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:59.322996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e80ea6-1a18-46a3-86b7-a641319ed702 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:59.365991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 276/556] 10.222.0.79 () {56 vars in 1135 bytes} [Mon Jul 4 09:55:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:55:59.372436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de59241-e54d-4d68-add0-34bd80841166 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:59.372726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de59241-e54d-4d68-add0-34bd80841166 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:59.373006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de59241-e54d-4d68-add0-34bd80841166 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:59.375145 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 277/557] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:55:59.375852 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de59241-e54d-4d68-add0-34bd80841166 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:59.417435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa97ae9-07e2-4812-89c1-552635cb0871 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:59.447724 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:59.459641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6032e-79c7-47c6-a045-be4d599c38f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:59.460685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6032e-79c7-47c6-a045-be4d599c38f1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:59.461217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6032e-79c7-47c6-a045-be4d599c38f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:59.461720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6032e-79c7-47c6-a045-be4d599c38f1 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:59.462774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6032e-79c7-47c6-a045-be4d599c38f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:55:59.463300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6032e-79c7-47c6-a045-be4d599c38f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:59.465240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6032e-79c7-47c6-a045-be4d599c38f1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:59.484631 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de59241-e54d-4d68-add0-34bd80841166 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:59.491178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 281/558] 10.222.0.79 () {56 vars in 1133 bytes} [Mon Jul 4 09:55:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:55:59.493049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:59.493964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a33f0-abe0-43e2-8a1a-3c06cbc018db None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:59.494462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a33f0-abe0-43e2-8a1a-3c06cbc018db None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:59.494963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a33f0-abe0-43e2-8a1a-3c06cbc018db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:59.514054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1c4a33f0-abe0-43e2-8a1a-3c06cbc018db None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:55:59.515602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1c4a33f0-abe0-43e2-8a1a-3c06cbc018db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:55:59.521630 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb08eff-6033-44d3-bde6-1c30f7488c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:59.522369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb08eff-6033-44d3-bde6-1c30f7488c18 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:59.522647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb08eff-6033-44d3-bde6-1c30f7488c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:59.522929 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb08eff-6033-44d3-bde6-1c30f7488c18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:59.592224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 282/559] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:55:59.593424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a33f0-abe0-43e2-8a1a-3c06cbc018db None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:59.602871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb08eff-6033-44d3-bde6-1c30f7488c18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:59.604362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb08eff-6033-44d3-bde6-1c30f7488c18 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:59.704886 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a33f0-abe0-43e2-8a1a-3c06cbc018db None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:59.726337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 278/560] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:55:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:55:59.726337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb08eff-6033-44d3-bde6-1c30f7488c18 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:59.729960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:59.731920 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:59.750393 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42422712-99b9-4581-b76e-7c6f3724e87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:59.751433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42422712-99b9-4581-b76e-7c6f3724e87c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:59.751722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42422712-99b9-4581-b76e-7c6f3724e87c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:59.752167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42422712-99b9-4581-b76e-7c6f3724e87c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:59.763110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42422712-99b9-4581-b76e-7c6f3724e87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:59.764588 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42422712-99b9-4581-b76e-7c6f3724e87c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-42422712-99b9-4581-b76e-7c6f3724e87c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:59.784253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:59.785407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:59.786497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:59.786497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:59.786497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:59.786497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:59.786497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 283/561] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:59.786788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42422712-99b9-4581-b76e-7c6f3724e87c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:59.833767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb08eff-6033-44d3-bde6-1c30f7488c18 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:59.869209 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0123a-1de1-4215-ba42-c315fa37aa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:59.870130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0123a-1de1-4215-ba42-c315fa37aa90 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:59.870444 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0123a-1de1-4215-ba42-c315fa37aa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:59.870732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0123a-1de1-4215-ba42-c315fa37aa90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:59.882103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0123a-1de1-4215-ba42-c315fa37aa90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d5f0123a-1de1-4215-ba42-c315fa37aa90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:55:59.890241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:59.891397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:59.892225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:59.892225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:59.892225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:59.892225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:59.892225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:55:59.892225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:55:59.892225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:55:59.892225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:55:59.892225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:55:59.892225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:55:59.892225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:55:59.892225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:55:59.892225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 279/562] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:55:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:59.892913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4795f8d-5402-4f9d-b66e-262884cf28d0 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:59.893453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4795f8d-5402-4f9d-b66e-262884cf28d0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:59.893453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4795f8d-5402-4f9d-b66e-262884cf28d0 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:59.894778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 280/563] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:55:59.899664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4795f8d-5402-4f9d-b66e-262884cf28d0 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:55:59.902380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42422712-99b9-4581-b76e-7c6f3724e87c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:55:59.939160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186394e-f98b-45c5-ba74-8c47340b196e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:55:59.940314 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186394e-f98b-45c5-ba74-8c47340b196e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:55:59.940795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186394e-f98b-45c5-ba74-8c47340b196e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:55:59.941263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186394e-f98b-45c5-ba74-8c47340b196e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:55:59.952268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186394e-f98b-45c5-ba74-8c47340b196e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:55:59.954183 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186394e-f98b-45c5-ba74-8c47340b196e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:55:59.968520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-0186394e-f98b-45c5-ba74-8c47340b196e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:55:59.974851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:55:59.976128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:55:59.977163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:55:59.977163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:55:59.977163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:55:59.977163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:55:59.977163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 284/564] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:55:59.978049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186394e-f98b-45c5-ba74-8c47340b196e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:00.002129 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:00.013823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4795f8d-5402-4f9d-b66e-262884cf28d0 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:00.049460 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a5fba-a539-43a8-97be-83f126c20705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:00.050175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a5fba-a539-43a8-97be-83f126c20705 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.050532 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a5fba-a539-43a8-97be-83f126c20705 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.050813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a5fba-a539-43a8-97be-83f126c20705 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.051536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a5fba-a539-43a8-97be-83f126c20705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:00.051848 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a5fba-a539-43a8-97be-83f126c20705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:00.053222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a5fba-a539-43a8-97be-83f126c20705 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:00.065671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 281/565] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:00.066983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e128ade-a5f1-4634-962d-2ceebbb5bcd1 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.067271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e128ade-a5f1-4634-962d-2ceebbb5bcd1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.067548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e128ade-a5f1-4634-962d-2ceebbb5bcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.084463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5e128ade-a5f1-4634-962d-2ceebbb5bcd1 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:00.085685 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5e128ade-a5f1-4634-962d-2ceebbb5bcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:00.105421 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186394e-f98b-45c5-ba74-8c47340b196e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:00.145751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44ccf7-2f72-46c8-a39d-ace8e82aebde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:00.146436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44ccf7-2f72-46c8-a39d-ace8e82aebde None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.146719 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44ccf7-2f72-46c8-a39d-ace8e82aebde None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.147018 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44ccf7-2f72-46c8-a39d-ace8e82aebde None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.147766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44ccf7-2f72-46c8-a39d-ace8e82aebde None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:00.148074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44ccf7-2f72-46c8-a39d-ace8e82aebde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:00.149531 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44ccf7-2f72-46c8-a39d-ace8e82aebde None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:00.156984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 282/566] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:00.161804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 285/567] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:55:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:00.161944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e128ade-a5f1-4634-962d-2ceebbb5bcd1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:00.162036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd492946-7acd-4b6e-bf59-666a6ba9b185 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.162036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd492946-7acd-4b6e-bf59-666a6ba9b185 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.162036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd492946-7acd-4b6e-bf59-666a6ba9b185 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.162036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 286/568] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:00.162036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd492946-7acd-4b6e-bf59-666a6ba9b185 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:00.231468 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:00.245175 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:00.264806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd492946-7acd-4b6e-bf59-666a6ba9b185 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:00.275346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e128ade-a5f1-4634-962d-2ceebbb5bcd1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:00.297397 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fbe6a-de15-438a-99e5-8cb7f28b1357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:00.298226 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fbe6a-de15-438a-99e5-8cb7f28b1357 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.298508 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fbe6a-de15-438a-99e5-8cb7f28b1357 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.298775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fbe6a-de15-438a-99e5-8cb7f28b1357 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.305188 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fbe6a-de15-438a-99e5-8cb7f28b1357 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:00.306673 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fbe6a-de15-438a-99e5-8cb7f28b1357 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:00.310461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db3fa9-2057-4cb8-9606-244f1bf09070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:00.311135 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db3fa9-2057-4cb8-9606-244f1bf09070 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.311423 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db3fa9-2057-4cb8-9606-244f1bf09070 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.311702 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db3fa9-2057-4cb8-9606-244f1bf09070 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.312427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db3fa9-2057-4cb8-9606-244f1bf09070 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:00.312865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db3fa9-2057-4cb8-9606-244f1bf09070 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:00.314427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db3fa9-2057-4cb8-9606-244f1bf09070 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-cf5fbe6a-de15-438a-99e5-8cb7f28b1357 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:00.320348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:00.321262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:00.322202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:00.322202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:00.322202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:00.322202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:00.322202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:00.322202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:00.322202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:00.322202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:00.322202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 287/569] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:00.322684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fbe6a-de15-438a-99e5-8cb7f28b1357 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:00.328744 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 283/570] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:00.330228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfe4e1-0be7-4d58-b1e5-850827921eec None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.330519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfe4e1-0be7-4d58-b1e5-850827921eec None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.330793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfe4e1-0be7-4d58-b1e5-850827921eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.348866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c9dfe4e1-0be7-4d58-b1e5-850827921eec None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:00.350103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c9dfe4e1-0be7-4d58-b1e5-850827921eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:00.417151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 284/571] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:00.418230 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfe4e1-0be7-4d58-b1e5-850827921eec None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:00.435489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fbe6a-de15-438a-99e5-8cb7f28b1357 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:00.466783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:00.472097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff62eb2-6836-4e67-ba08-24c7691046df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:00.472907 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff62eb2-6836-4e67-ba08-24c7691046df None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.473192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff62eb2-6836-4e67-ba08-24c7691046df None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.473466 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff62eb2-6836-4e67-ba08-24c7691046df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.483803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff62eb2-6836-4e67-ba08-24c7691046df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:00.485256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff62eb2-6836-4e67-ba08-24c7691046df None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:00.486927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-aff62eb2-6836-4e67-ba08-24c7691046df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:00.503863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:00.504899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:00.505939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:00.505939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:00.505939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:00.505939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:00.505939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 288/572] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:00.506344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff62eb2-6836-4e67-ba08-24c7691046df None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:00.525664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfe4e1-0be7-4d58-b1e5-850827921eec None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:00.560389 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e43d70-7a13-4991-93fc-0ba3f303b1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:00.561189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e43d70-7a13-4991-93fc-0ba3f303b1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.561467 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e43d70-7a13-4991-93fc-0ba3f303b1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.561770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e43d70-7a13-4991-93fc-0ba3f303b1a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.571868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e43d70-7a13-4991-93fc-0ba3f303b1a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-58e43d70-7a13-4991-93fc-0ba3f303b1a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:00.579537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:00.580566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:00.581406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:00.581406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:00.581406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:00.581406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:00.581406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:00.581406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:00.581406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:00.581406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:00.581406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:00.581406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:00.581406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:00.581406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:00.581406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 285/573] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:56:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:00.582757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e43d70-7a13-4991-93fc-0ba3f303b1a2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:00.607687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff62eb2-6836-4e67-ba08-24c7691046df None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:00.640926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b56412-5581-4838-8c4b-478d7d8de5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:00.641592 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b56412-5581-4838-8c4b-478d7d8de5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.641892 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b56412-5581-4838-8c4b-478d7d8de5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.642170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b56412-5581-4838-8c4b-478d7d8de5e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.642957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b56412-5581-4838-8c4b-478d7d8de5e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:00.643261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b56412-5581-4838-8c4b-478d7d8de5e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:00.644698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b56412-5581-4838-8c4b-478d7d8de5e3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:00.656079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 289/574] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:00.657082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b56412-5581-4838-8c4b-478d7d8de5e3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:00.683261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e43d70-7a13-4991-93fc-0ba3f303b1a2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:00.690617 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:00.715166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:00.718687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba0390-6617-4dea-8891-50cce89e18d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:00.719523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba0390-6617-4dea-8891-50cce89e18d4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.719804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba0390-6617-4dea-8891-50cce89e18d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.720104 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba0390-6617-4dea-8891-50cce89e18d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.727456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba0390-6617-4dea-8891-50cce89e18d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:00.728924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba0390-6617-4dea-8891-50cce89e18d4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-8aba0390-6617-4dea-8891-50cce89e18d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:00.743049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:00.744035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:00.744903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:00.744903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:00.744903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:00.744903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:00.744903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:00.744903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:00.744903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:00.744903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:00.744903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 286/575] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:00.745381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba0390-6617-4dea-8891-50cce89e18d4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:00.768212 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b56412-5581-4838-8c4b-478d7d8de5e3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:00.801905 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8269b0a-ca91-4313-b2d0-2e6b9686a24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:00.802710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8269b0a-ca91-4313-b2d0-2e6b9686a24e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.802994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8269b0a-ca91-4313-b2d0-2e6b9686a24e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.803272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8269b0a-ca91-4313-b2d0-2e6b9686a24e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.814814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8269b0a-ca91-4313-b2d0-2e6b9686a24e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-f8269b0a-ca91-4313-b2d0-2e6b9686a24e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:00.822477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:00.823378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:00.824286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:00.824286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:00.824286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:00.824286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:00.824286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:00.824286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:00.824286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:00.824286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:00.824286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:00.824286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:00.824286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:00.824286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:00.824286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 290/576] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:56:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:00.824989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8269b0a-ca91-4313-b2d0-2e6b9686a24e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:00.850116 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba0390-6617-4dea-8891-50cce89e18d4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:00.884193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461c595-6121-4715-8ba3-895130532093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:00.884865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461c595-6121-4715-8ba3-895130532093 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.885157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461c595-6121-4715-8ba3-895130532093 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.885439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461c595-6121-4715-8ba3-895130532093 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.886196 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461c595-6121-4715-8ba3-895130532093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:00.886506 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461c595-6121-4715-8ba3-895130532093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:00.887904 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461c595-6121-4715-8ba3-895130532093 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:00.899102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 287/577] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:00.900036 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461c595-6121-4715-8ba3-895130532093 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:00.921437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8269b0a-ca91-4313-b2d0-2e6b9686a24e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:00.928790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:00.935030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:00.956296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0520d63-0c7d-4244-9d54-44a767c03127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:00.956964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0520d63-0c7d-4244-9d54-44a767c03127 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:00.957230 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0520d63-0c7d-4244-9d54-44a767c03127 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:00.957504 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0520d63-0c7d-4244-9d54-44a767c03127 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:00.958288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0520d63-0c7d-4244-9d54-44a767c03127 None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:00.958600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0520d63-0c7d-4244-9d54-44a767c03127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:00.960009 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0520d63-0c7d-4244-9d54-44a767c03127 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:00.967814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 291/578] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:56:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:00.968595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0520d63-0c7d-4244-9d54-44a767c03127 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:01.001423 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461c595-6121-4715-8ba3-895130532093 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:01.033840 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c065d47-6e16-4b6b-afca-a44494554ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:01.034726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c065d47-6e16-4b6b-afca-a44494554ce2 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:01.035045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c065d47-6e16-4b6b-afca-a44494554ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:01.035317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c065d47-6e16-4b6b-afca-a44494554ce2 None admin] PATH_INFO: `/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/ffc2409cb7fc4373a74ddfb48d501d30/roles/fc6661dcb3994755a5c238bf37e17bb9` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:01.049163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c065d47-6e16-4b6b-afca-a44494554ce2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f32306262e8744eab4bdc6d28e24e6d0, user_id=ffc2409cb7fc4373a74ddfb48d501d30, role_id=fc6661dcb3994755a5c238bf37e17bb9)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:01.050630 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c065d47-6e16-4b6b-afca-a44494554ce2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:01.064291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 288/579] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:56:00 2022] PUT /identity/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/ffc2409cb7fc4373a74ddfb48d501d30/roles/fc6661dcb3994755a5c238bf37e17bb9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:56:01.065104 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c065d47-6e16-4b6b-afca-a44494554ce2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:01.066349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0520d63-0c7d-4244-9d54-44a767c03127 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:01.108042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af074fc0-0c90-4988-9629-b6cbde4ddfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:01.108728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af074fc0-0c90-4988-9629-b6cbde4ddfd1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:01.109012 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af074fc0-0c90-4988-9629-b6cbde4ddfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:01.109341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af074fc0-0c90-4988-9629-b6cbde4ddfd1 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:01.110812 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af074fc0-0c90-4988-9629-b6cbde4ddfd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:01.111126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af074fc0-0c90-4988-9629-b6cbde4ddfd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:01.112633 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af074fc0-0c90-4988-9629-b6cbde4ddfd1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:01.135114 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 292/580] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:01.135114 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af074fc0-0c90-4988-9629-b6cbde4ddfd1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:01.160408 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:01.179964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:01.189996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c065d47-6e16-4b6b-afca-a44494554ce2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:01.229558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085bb17-5b2e-43a3-a308-b6d4a538f3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:01.234171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085bb17-5b2e-43a3-a308-b6d4a538f3aa None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:01.234602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085bb17-5b2e-43a3-a308-b6d4a538f3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:01.234884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085bb17-5b2e-43a3-a308-b6d4a538f3aa None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:01.235755 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085bb17-5b2e-43a3-a308-b6d4a538f3aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:01.236069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085bb17-5b2e-43a3-a308-b6d4a538f3aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:01.237467 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085bb17-5b2e-43a3-a308-b6d4a538f3aa None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:01.247227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 289/581] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:56:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:01.252826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085bb17-5b2e-43a3-a308-b6d4a538f3aa None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:01.266088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af074fc0-0c90-4988-9629-b6cbde4ddfd1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:01.307727 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73374e73-5b85-4bfb-806f-35c96a06751f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:01.308529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73374e73-5b85-4bfb-806f-35c96a06751f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:01.308867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73374e73-5b85-4bfb-806f-35c96a06751f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:01.309143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73374e73-5b85-4bfb-806f-35c96a06751f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:01.316696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73374e73-5b85-4bfb-806f-35c96a06751f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:01.318193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73374e73-5b85-4bfb-806f-35c96a06751f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-73374e73-5b85-4bfb-806f-35c96a06751f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:01.332948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:01.333971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:01.335821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:01.335821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:01.335821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:01.335821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:01.335821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:01.335821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:01.335821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:01.335821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:01.335821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:56:01.335821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:01.335821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 293/582] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:56:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:01.335821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73374e73-5b85-4bfb-806f-35c96a06751f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:01.360590 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085bb17-5b2e-43a3-a308-b6d4a538f3aa None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:01.397044 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5b6c8-842b-4bb6-b189-1fd544cca90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:01.397889 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5b6c8-842b-4bb6-b189-1fd544cca90b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:01.398176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5b6c8-842b-4bb6-b189-1fd544cca90b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:01.398452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5b6c8-842b-4bb6-b189-1fd544cca90b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:01.405724 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:01.409954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5b6c8-842b-4bb6-b189-1fd544cca90b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:01.411908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5b6c8-842b-4bb6-b189-1fd544cca90b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-28e5b6c8-842b-4bb6-b189-1fd544cca90b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:01.431326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:01.432425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:01.433446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:01.433446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:01.433446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:01.433446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:01.435011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 290/583] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:01.436071 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5b6c8-842b-4bb6-b189-1fd544cca90b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:01.441652 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:01.447858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73374e73-5b85-4bfb-806f-35c96a06751f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:01.483133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee748cb-82ca-4a47-8aec-9d56198354f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:01.483788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee748cb-82ca-4a47-8aec-9d56198354f9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:01.484074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee748cb-82ca-4a47-8aec-9d56198354f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:01.484484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee748cb-82ca-4a47-8aec-9d56198354f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:01.485737 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee748cb-82ca-4a47-8aec-9d56198354f9 None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=7ec37893ec8c480990c17cb03cb0ca19) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:01.486051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee748cb-82ca-4a47-8aec-9d56198354f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:01.487502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee748cb-82ca-4a47-8aec-9d56198354f9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:01.493126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 294/584] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:01 2022] GET /identity/v3/role_assignments?user.id=7ec37893ec8c480990c17cb03cb0ca19&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:01.493939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee748cb-82ca-4a47-8aec-9d56198354f9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:01.566947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5b6c8-842b-4bb6-b189-1fd544cca90b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:01.612272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73df0c8-52fe-476a-92d6-e2e131805901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:01.614111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73df0c8-52fe-476a-92d6-e2e131805901 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:01.614912 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73df0c8-52fe-476a-92d6-e2e131805901 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:01.615405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73df0c8-52fe-476a-92d6-e2e131805901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:01.626454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee748cb-82ca-4a47-8aec-9d56198354f9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:01.632542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73df0c8-52fe-476a-92d6-e2e131805901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:01.634055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73df0c8-52fe-476a-92d6-e2e131805901 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d73df0c8-52fe-476a-92d6-e2e131805901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:01.652764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:01.653769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:01.654661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:01.654661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:01.654661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:01.654661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:01.659078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 291/585] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:01.660099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73df0c8-52fe-476a-92d6-e2e131805901 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:01.675865 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127b411-810e-4c37-aa28-7c10db767cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:01.677180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127b411-810e-4c37-aa28-7c10db767cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:01.678161 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127b411-810e-4c37-aa28-7c10db767cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:01.678648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127b411-810e-4c37-aa28-7c10db767cb6 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:01.679937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127b411-810e-4c37-aa28-7c10db767cb6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:01.680681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127b411-810e-4c37-aa28-7c10db767cb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:01.683136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127b411-810e-4c37-aa28-7c10db767cb6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:01.707507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:01.709868 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 295/586] 10.222.0.79 () {56 vars in 1137 bytes} [Mon Jul 4 09:56:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:01.711729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127b411-810e-4c37-aa28-7c10db767cb6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:01.746045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:01.775299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73df0c8-52fe-476a-92d6-e2e131805901 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:01.812178 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6db2cd-5807-4a79-a40f-894764c965bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:01.813017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6db2cd-5807-4a79-a40f-894764c965bb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:01.813440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6db2cd-5807-4a79-a40f-894764c965bb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:01.813907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6db2cd-5807-4a79-a40f-894764c965bb None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:01.814748 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6db2cd-5807-4a79-a40f-894764c965bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:01.815128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6db2cd-5807-4a79-a40f-894764c965bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:01.816718 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6db2cd-5807-4a79-a40f-894764c965bb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:01.833967 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 292/587] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:01.834974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6db2cd-5807-4a79-a40f-894764c965bb None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:01.837475 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127b411-810e-4c37-aa28-7c10db767cb6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:01.884244 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab4c29-bc4e-4c1f-af28-0c2eb84d5908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:01.885784 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab4c29-bc4e-4c1f-af28-0c2eb84d5908 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:01.886082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab4c29-bc4e-4c1f-af28-0c2eb84d5908 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:01.886365 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab4c29-bc4e-4c1f-af28-0c2eb84d5908 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:01.887099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab4c29-bc4e-4c1f-af28-0c2eb84d5908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:01.887434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab4c29-bc4e-4c1f-af28-0c2eb84d5908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:01.888832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab4c29-bc4e-4c1f-af28-0c2eb84d5908 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:01.900677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 296/588] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:01.901604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab4c29-bc4e-4c1f-af28-0c2eb84d5908 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:01.958613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6db2cd-5807-4a79-a40f-894764c965bb None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:01.961243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:01.985090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:01.998392 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c64792-24d0-464f-a0d5-20a12300606a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:01.999482 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c64792-24d0-464f-a0d5-20a12300606a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:01.999912 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c64792-24d0-464f-a0d5-20a12300606a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:02.000630 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c64792-24d0-464f-a0d5-20a12300606a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:02.013155 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c64792-24d0-464f-a0d5-20a12300606a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:02.014327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab4c29-bc4e-4c1f-af28-0c2eb84d5908 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:02.015427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c64792-24d0-464f-a0d5-20a12300606a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-83c64792-24d0-464f-a0d5-20a12300606a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:02.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:02.036222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:02.037095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:02.037095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:02.037095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:02.037095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:02.039391 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 293/589] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:02.040472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c64792-24d0-464f-a0d5-20a12300606a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:02.059165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65f5ec-54e5-4ceb-ad15-d2119d40d3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:02.060428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65f5ec-54e5-4ceb-ad15-d2119d40d3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:02.060912 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65f5ec-54e5-4ceb-ad15-d2119d40d3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:02.061423 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65f5ec-54e5-4ceb-ad15-d2119d40d3a6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:02.070122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65f5ec-54e5-4ceb-ad15-d2119d40d3a6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:02.071958 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65f5ec-54e5-4ceb-ad15-d2119d40d3a6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-7a65f5ec-54e5-4ceb-ad15-d2119d40d3a6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:02.087064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:02.088030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:02.088967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:02.088967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:02.088967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:02.088967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:02.088967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:02.088967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:02.088967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:02.088967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:02.088967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:56:02.088967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:02.091804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 297/590] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:56:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:02.092838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65f5ec-54e5-4ceb-ad15-d2119d40d3a6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:02.162717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c64792-24d0-464f-a0d5-20a12300606a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:02.199659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754ad39-3dd7-492f-bf00-6e1641ff6a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:02.200592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754ad39-3dd7-492f-bf00-6e1641ff6a82 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:02.200773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754ad39-3dd7-492f-bf00-6e1641ff6a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:02.201016 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754ad39-3dd7-492f-bf00-6e1641ff6a82 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:02.208868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754ad39-3dd7-492f-bf00-6e1641ff6a82 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:02.210277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754ad39-3dd7-492f-bf00-6e1641ff6a82 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:02.222447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65f5ec-54e5-4ceb-ad15-d2119d40d3a6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-7754ad39-3dd7-492f-bf00-6e1641ff6a82 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:02.226159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:02.227079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:02.228572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:02.228572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:02.228572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:02.228572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:02.228572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:02.228572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:02.228572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:02.228572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:02.228572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:56:02.228572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:02.228572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 294/591] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:56:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:02.228572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754ad39-3dd7-492f-bf00-6e1641ff6a82 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:02.245021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:02.266181 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729a3e4-3f7e-4490-b62f-1b5d9f65e512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:02.267253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729a3e4-3f7e-4490-b62f-1b5d9f65e512 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:02.267855 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729a3e4-3f7e-4490-b62f-1b5d9f65e512 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:02.268333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729a3e4-3f7e-4490-b62f-1b5d9f65e512 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:02.269280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729a3e4-3f7e-4490-b62f-1b5d9f65e512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:02.269857 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729a3e4-3f7e-4490-b62f-1b5d9f65e512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:02.271623 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729a3e4-3f7e-4490-b62f-1b5d9f65e512 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:02.279588 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:02.286059 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 298/592] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:02.287369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729a3e4-3f7e-4490-b62f-1b5d9f65e512 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:02.347546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754ad39-3dd7-492f-bf00-6e1641ff6a82 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:02.383779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c062c-82b5-4b24-b2fc-ad4a4915ed53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:02.384464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c062c-82b5-4b24-b2fc-ad4a4915ed53 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:02.384747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c062c-82b5-4b24-b2fc-ad4a4915ed53 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:02.385024 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c062c-82b5-4b24-b2fc-ad4a4915ed53 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:02.385903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c062c-82b5-4b24-b2fc-ad4a4915ed53 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:02.386214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c062c-82b5-4b24-b2fc-ad4a4915ed53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:02.387684 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c062c-82b5-4b24-b2fc-ad4a4915ed53 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:02.400105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729a3e4-3f7e-4490-b62f-1b5d9f65e512 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:02.405878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 295/593] 10.222.0.79 () {56 vars in 1139 bytes} [Mon Jul 4 09:56:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:02.407365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c062c-82b5-4b24-b2fc-ad4a4915ed53 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:02.439085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45920b-2613-48c8-b89b-7dd2bfe63cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:02.440151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45920b-2613-48c8-b89b-7dd2bfe63cbc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:02.440691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45920b-2613-48c8-b89b-7dd2bfe63cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:02.441165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45920b-2613-48c8-b89b-7dd2bfe63cbc None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:02.442335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45920b-2613-48c8-b89b-7dd2bfe63cbc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:02.442936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45920b-2613-48c8-b89b-7dd2bfe63cbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:02.444731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45920b-2613-48c8-b89b-7dd2bfe63cbc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:02.465071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 299/594] 10.222.0.79 () {56 vars in 1137 bytes} [Mon Jul 4 09:56:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:02.466272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45920b-2613-48c8-b89b-7dd2bfe63cbc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:02.479273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:02.510918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c062c-82b5-4b24-b2fc-ad4a4915ed53 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:02.531530 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:02.546529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a070d-f451-468f-a3fb-62b1294fa943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:02.548054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a070d-f451-468f-a3fb-62b1294fa943 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:02.548341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a070d-f451-468f-a3fb-62b1294fa943 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:02.548629 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a070d-f451-468f-a3fb-62b1294fa943 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:02.555938 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a070d-f451-468f-a3fb-62b1294fa943 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:02.557415 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a070d-f451-468f-a3fb-62b1294fa943 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-764a070d-f451-468f-a3fb-62b1294fa943 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:02.571946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:02.572830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:02.574438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:02.574438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:02.574438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:02.574438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:02.574438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:02.574438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:02.574438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:02.574438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:02.574438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 296/595] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:02.574438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a070d-f451-468f-a3fb-62b1294fa943 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:02.586773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45920b-2613-48c8-b89b-7dd2bfe63cbc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:02.624144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc5a18-33cf-4f7f-924e-5ecdab608820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:02.625109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc5a18-33cf-4f7f-924e-5ecdab608820 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:02.625846 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc5a18-33cf-4f7f-924e-5ecdab608820 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:02.625846 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc5a18-33cf-4f7f-924e-5ecdab608820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:02.636442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc5a18-33cf-4f7f-924e-5ecdab608820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:02.638123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc5a18-33cf-4f7f-924e-5ecdab608820 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-fcbc5a18-33cf-4f7f-924e-5ecdab608820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:02.657738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:02.658815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:02.659736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:02.659736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:02.659736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:02.659736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:02.659978 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 300/596] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:02.661180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc5a18-33cf-4f7f-924e-5ecdab608820 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:02.697733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a070d-f451-468f-a3fb-62b1294fa943 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:02.734133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:02.739945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0837c-7383-46f0-b616-24077ae93052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:02.740846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0837c-7383-46f0-b616-24077ae93052 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:02.741129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0837c-7383-46f0-b616-24077ae93052 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:02.741402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0837c-7383-46f0-b616-24077ae93052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:02.754389 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0837c-7383-46f0-b616-24077ae93052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:02.756172 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0837c-7383-46f0-b616-24077ae93052 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-bdb0837c-7383-46f0-b616-24077ae93052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:02.774159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:02.775188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:02.776075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:02.776075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:02.776075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:02.776075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:02.777872 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc5a18-33cf-4f7f-924e-5ecdab608820 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:02.780026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 297/597] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:02.781879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0837c-7383-46f0-b616-24077ae93052 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:02.789152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:02.821088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409fcee-0a6a-4a94-a2b3-5c8cbd64e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:02.821948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409fcee-0a6a-4a94-a2b3-5c8cbd64e345 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:02.822334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409fcee-0a6a-4a94-a2b3-5c8cbd64e345 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:02.822710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409fcee-0a6a-4a94-a2b3-5c8cbd64e345 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:02.823641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409fcee-0a6a-4a94-a2b3-5c8cbd64e345 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:02.824054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409fcee-0a6a-4a94-a2b3-5c8cbd64e345 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:02.826179 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409fcee-0a6a-4a94-a2b3-5c8cbd64e345 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:02.841421 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 301/598] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:02.842481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409fcee-0a6a-4a94-a2b3-5c8cbd64e345 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:02.890463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0837c-7383-46f0-b616-24077ae93052 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:02.927021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27bfa8c-e83d-4a63-8ecc-77ff6b56a694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:02.927948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bfa8c-e83d-4a63-8ecc-77ff6b56a694 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:02.928368 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bfa8c-e83d-4a63-8ecc-77ff6b56a694 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:02.928775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bfa8c-e83d-4a63-8ecc-77ff6b56a694 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:02.929646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27bfa8c-e83d-4a63-8ecc-77ff6b56a694 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:02.930086 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27bfa8c-e83d-4a63-8ecc-77ff6b56a694 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:02.931615 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27bfa8c-e83d-4a63-8ecc-77ff6b56a694 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:02.944163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 298/599] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:02.945229 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27bfa8c-e83d-4a63-8ecc-77ff6b56a694 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:02.980849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409fcee-0a6a-4a94-a2b3-5c8cbd64e345 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:02.994246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:03.038938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f4543-fb02-45f8-9d4b-ce1a47a13167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:03.039796 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f4543-fb02-45f8-9d4b-ce1a47a13167 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:03.040211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f4543-fb02-45f8-9d4b-ce1a47a13167 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:03.040972 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f4543-fb02-45f8-9d4b-ce1a47a13167 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:03.042198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f4543-fb02-45f8-9d4b-ce1a47a13167 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:03.042610 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f4543-fb02-45f8-9d4b-ce1a47a13167 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:03.044827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f4543-fb02-45f8-9d4b-ce1a47a13167 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:03.066355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 302/600] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:03.066355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f4543-fb02-45f8-9d4b-ce1a47a13167 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:03.068817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27bfa8c-e83d-4a63-8ecc-77ff6b56a694 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:03.099886 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:03.107823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ac077-0ce9-4de6-af45-3db1a16ed12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:03.109072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac077-0ce9-4de6-af45-3db1a16ed12b None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:03.109571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac077-0ce9-4de6-af45-3db1a16ed12b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:03.110099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac077-0ce9-4de6-af45-3db1a16ed12b None admin] PATH_INFO: `/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/fef685a17c804e2bbfea1e07cfdf5cfe/roles/fc6661dcb3994755a5c238bf37e17bb9` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:03.131119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ac077-0ce9-4de6-af45-3db1a16ed12b None admin] RBAC: Authorizing `identity:create_grant(project_id=f32306262e8744eab4bdc6d28e24e6d0, user_id=fef685a17c804e2bbfea1e07cfdf5cfe, role_id=fc6661dcb3994755a5c238bf37e17bb9)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:03.133138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ac077-0ce9-4de6-af45-3db1a16ed12b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:03.149644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 299/601] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:56:02 2022] PUT /identity/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/fef685a17c804e2bbfea1e07cfdf5cfe/roles/fc6661dcb3994755a5c238bf37e17bb9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:56:03.150391 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ac077-0ce9-4de6-af45-3db1a16ed12b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:03.215380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f4543-fb02-45f8-9d4b-ce1a47a13167 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:03.261195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:03.269439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ac077-0ce9-4de6-af45-3db1a16ed12b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:03.270391 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1ada6-661a-4d9f-9f01-e9d75a2deed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:03.271387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1ada6-661a-4d9f-9f01-e9d75a2deed5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:03.272588 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1ada6-661a-4d9f-9f01-e9d75a2deed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:03.273055 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1ada6-661a-4d9f-9f01-e9d75a2deed5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:03.284244 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1ada6-661a-4d9f-9f01-e9d75a2deed5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:03.286844 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1ada6-661a-4d9f-9f01-e9d75a2deed5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-2bd1ada6-661a-4d9f-9f01-e9d75a2deed5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:03.307607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:03.308575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:03.309463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:03.309463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:03.309463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:03.309463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:03.309463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:03.309463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:03.309463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:03.309463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:03.312186 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 303/602] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:03.314805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec25090-c196-4fe6-b18a-870e3a184d7a None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:03.315154 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d541d-2aeb-48a9-b01e-47fa27ee2b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:03.315542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec25090-c196-4fe6-b18a-870e3a184d7a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:03.315998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec25090-c196-4fe6-b18a-870e3a184d7a None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:03.316101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d541d-2aeb-48a9-b01e-47fa27ee2b9a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:03.316462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d541d-2aeb-48a9-b01e-47fa27ee2b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:03.316793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d541d-2aeb-48a9-b01e-47fa27ee2b9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:03.318495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 304/603] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:03.319724 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec25090-c196-4fe6-b18a-870e3a184d7a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:03.328262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d541d-2aeb-48a9-b01e-47fa27ee2b9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:03.328262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d541d-2aeb-48a9-b01e-47fa27ee2b9a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d01d541d-2aeb-48a9-b01e-47fa27ee2b9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:03.347410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:03.348340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:03.349901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:03.349901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:03.349901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:03.349901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:03.349901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:03.349901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:03.349901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:03.349901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:03.349901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 300/604] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:03.351367 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ab08d-7e63-4962-96fc-534faf9fee9e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:03.351703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ab08d-7e63-4962-96fc-534faf9fee9e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:03.351980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ab08d-7e63-4962-96fc-534faf9fee9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:03.370796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-dd1ab08d-7e63-4962-96fc-534faf9fee9e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:03.371976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-dd1ab08d-7e63-4962-96fc-534faf9fee9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:03.442275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 301/605] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:03.443120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ab08d-7e63-4962-96fc-534faf9fee9e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:03.445884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:03.460925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec25090-c196-4fe6-b18a-870e3a184d7a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:03.498809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f96b6-ef54-4a28-8cd2-ee4482b05bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:03.499884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f96b6-ef54-4a28-8cd2-ee4482b05bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:03.500492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f96b6-ef54-4a28-8cd2-ee4482b05bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:03.500967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f96b6-ef54-4a28-8cd2-ee4482b05bf9 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:03.502085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f96b6-ef54-4a28-8cd2-ee4482b05bf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:03.502644 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f96b6-ef54-4a28-8cd2-ee4482b05bf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:03.504356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f96b6-ef54-4a28-8cd2-ee4482b05bf9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:03.518787 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 305/606] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:03.520078 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f96b6-ef54-4a28-8cd2-ee4482b05bf9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:03.530797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:03.549021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ab08d-7e63-4962-96fc-534faf9fee9e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:03.584514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc2d4c-e012-48c6-ac25-1b6572170183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:03.585208 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc2d4c-e012-48c6-ac25-1b6572170183 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:03.585489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc2d4c-e012-48c6-ac25-1b6572170183 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:03.585790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc2d4c-e012-48c6-ac25-1b6572170183 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:03.586579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc2d4c-e012-48c6-ac25-1b6572170183 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:03.586887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc2d4c-e012-48c6-ac25-1b6572170183 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:03.588328 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc2d4c-e012-48c6-ac25-1b6572170183 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:03.601334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 302/607] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:03.602289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc2d4c-e012-48c6-ac25-1b6572170183 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:03.635164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f96b6-ef54-4a28-8cd2-ee4482b05bf9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:03.673904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9c21c-3851-4097-9ec7-4353abd98180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:03.674979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9c21c-3851-4097-9ec7-4353abd98180 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:03.674979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9c21c-3851-4097-9ec7-4353abd98180 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:03.675264 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9c21c-3851-4097-9ec7-4353abd98180 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:03.686584 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9c21c-3851-4097-9ec7-4353abd98180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-35c9c21c-3851-4097-9ec7-4353abd98180 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:03.694301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:03.695424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 306/608] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:56:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:03.696476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9c21c-3851-4097-9ec7-4353abd98180 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:03.709284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:03.709598 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc2d4c-e012-48c6-ac25-1b6572170183 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:03.746040 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad8f27-a281-46dd-aaa1-c157e2a374f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:03.746943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad8f27-a281-46dd-aaa1-c157e2a374f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:03.747230 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad8f27-a281-46dd-aaa1-c157e2a374f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:03.747510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad8f27-a281-46dd-aaa1-c157e2a374f7 None admin] PATH_INFO: `/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/3a1f11a4b8554a98b7626a82f01888c7/roles/fc6661dcb3994755a5c238bf37e17bb9` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:03.762997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad8f27-a281-46dd-aaa1-c157e2a374f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f32306262e8744eab4bdc6d28e24e6d0, user_id=3a1f11a4b8554a98b7626a82f01888c7, role_id=fc6661dcb3994755a5c238bf37e17bb9)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:03.764777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad8f27-a281-46dd-aaa1-c157e2a374f7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:03.767223 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:03.793648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 303/609] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:56:03 2022] PUT /identity/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/3a1f11a4b8554a98b7626a82f01888c7/roles/fc6661dcb3994755a5c238bf37e17bb9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:56:03.793648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656b2ca-7b4e-44a2-9697-80f468323408 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:03.793648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656b2ca-7b4e-44a2-9697-80f468323408 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:03.793648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656b2ca-7b4e-44a2-9697-80f468323408 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:03.793648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 304/610] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:03.793648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656b2ca-7b4e-44a2-9697-80f468323408 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:03.817845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9c21c-3851-4097-9ec7-4353abd98180 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:03.856021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0812486-329f-49e2-92c3-6e1a3a12d2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:03.857055 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0812486-329f-49e2-92c3-6e1a3a12d2b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:03.857405 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0812486-329f-49e2-92c3-6e1a3a12d2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:03.857781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0812486-329f-49e2-92c3-6e1a3a12d2b6 None admin] PATH_INFO: `/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/c5a4af83902d4370a832ed3ea3861b56/roles/fc6661dcb3994755a5c238bf37e17bb9` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:03.875461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0812486-329f-49e2-92c3-6e1a3a12d2b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f32306262e8744eab4bdc6d28e24e6d0, user_id=c5a4af83902d4370a832ed3ea3861b56, role_id=fc6661dcb3994755a5c238bf37e17bb9)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:03.877074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0812486-329f-49e2-92c3-6e1a3a12d2b6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:03.895128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 307/611] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:56:03 2022] PUT /identity/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/c5a4af83902d4370a832ed3ea3861b56/roles/fc6661dcb3994755a5c238bf37e17bb9 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:56:03.896854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32f477-4b9c-4576-a8cd-7089acd824d4 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:03.897235 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32f477-4b9c-4576-a8cd-7089acd824d4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:03.897643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32f477-4b9c-4576-a8cd-7089acd824d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:03.918088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7d32f477-4b9c-4576-a8cd-7089acd824d4 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:03.919498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7d32f477-4b9c-4576-a8cd-7089acd824d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:03.949845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656b2ca-7b4e-44a2-9697-80f468323408 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:04.005168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b90ef-81d3-4109-a24e-6b1cbc312c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:04.006305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b90ef-81d3-4109-a24e-6b1cbc312c9b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:04.007970 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b90ef-81d3-4109-a24e-6b1cbc312c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:04.008525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b90ef-81d3-4109-a24e-6b1cbc312c9b None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:04.009507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b90ef-81d3-4109-a24e-6b1cbc312c9b None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:04.010090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b90ef-81d3-4109-a24e-6b1cbc312c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:04.011704 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b90ef-81d3-4109-a24e-6b1cbc312c9b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:04.024059 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 305/612] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:56:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:04.030997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b90ef-81d3-4109-a24e-6b1cbc312c9b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:04.037939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:04.054215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 308/613] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:04.093130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32f477-4b9c-4576-a8cd-7089acd824d4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:04.107673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:04.166577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b90ef-81d3-4109-a24e-6b1cbc312c9b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:04.202692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ac6e4-0244-4730-bcb2-045d76f78923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:04.203844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ac6e4-0244-4730-bcb2-045d76f78923 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:04.204263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ac6e4-0244-4730-bcb2-045d76f78923 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:04.204743 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ac6e4-0244-4730-bcb2-045d76f78923 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:04.216195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ac6e4-0244-4730-bcb2-045d76f78923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:04.217860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ac6e4-0244-4730-bcb2-045d76f78923 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d52ac6e4-0244-4730-bcb2-045d76f78923 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:04.236671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:04.237833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:04.238907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:04.238907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:04.238907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:04.238907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:04.240897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 306/614] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:04.247531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ac6e4-0244-4730-bcb2-045d76f78923 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:04.265653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32f477-4b9c-4576-a8cd-7089acd824d4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:04.294258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40392ca0-010a-4460-944a-c8d34f85e537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:04.295101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40392ca0-010a-4460-944a-c8d34f85e537 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:04.295353 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40392ca0-010a-4460-944a-c8d34f85e537 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:04.295623 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40392ca0-010a-4460-944a-c8d34f85e537 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:04.306443 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40392ca0-010a-4460-944a-c8d34f85e537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-40392ca0-010a-4460-944a-c8d34f85e537 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:04.317650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:04.318799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:04.319801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:04.319801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:04.319801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:04.319801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:04.319801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:04.319801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:04.319801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:04.319801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:04.319801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:04.319801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:04.319801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:04.319801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:04.319801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 309/615] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:56:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:04.323058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40392ca0-010a-4460-944a-c8d34f85e537 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:04.401082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ac6e4-0244-4730-bcb2-045d76f78923 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:04.413259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:04.433260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:04.456688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc3ad2-c824-4cd9-9416-1d9ca18cd77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:04.457382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc3ad2-c824-4cd9-9416-1d9ca18cd77b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:04.457992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc3ad2-c824-4cd9-9416-1d9ca18cd77b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:04.458342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc3ad2-c824-4cd9-9416-1d9ca18cd77b None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:04.459574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc3ad2-c824-4cd9-9416-1d9ca18cd77b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:04.460090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc3ad2-c824-4cd9-9416-1d9ca18cd77b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:04.461876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc3ad2-c824-4cd9-9416-1d9ca18cd77b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:04.479818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 307/616] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:04.486681 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc3ad2-c824-4cd9-9416-1d9ca18cd77b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:04.531410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40392ca0-010a-4460-944a-c8d34f85e537 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:04.596197 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78919b30-615e-481e-98ca-fa0641a76046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:04.598181 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78919b30-615e-481e-98ca-fa0641a76046 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:04.598602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78919b30-615e-481e-98ca-fa0641a76046 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:04.599874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78919b30-615e-481e-98ca-fa0641a76046 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:04.601108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78919b30-615e-481e-98ca-fa0641a76046 None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:04.603786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78919b30-615e-481e-98ca-fa0641a76046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:04.605325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78919b30-615e-481e-98ca-fa0641a76046 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:04.620918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 310/617] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:56:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:04.626535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78919b30-615e-481e-98ca-fa0641a76046 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:04.646616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc3ad2-c824-4cd9-9416-1d9ca18cd77b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:04.681124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984bfe5-a7ae-4df8-9d2d-cc18da91aec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:04.682234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984bfe5-a7ae-4df8-9d2d-cc18da91aec5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:04.682659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984bfe5-a7ae-4df8-9d2d-cc18da91aec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:04.683069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984bfe5-a7ae-4df8-9d2d-cc18da91aec5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:04.690730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6984bfe5-a7ae-4df8-9d2d-cc18da91aec5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:04.692370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6984bfe5-a7ae-4df8-9d2d-cc18da91aec5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-6984bfe5-a7ae-4df8-9d2d-cc18da91aec5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:04.707163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:04.708278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:04.709269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:04.709269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:04.709269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:04.709269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:04.709269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:04.709269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:04.709269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:04.709269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:04.709269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:04.709269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:04.711678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 308/618] 10.222.0.79 () {56 vars in 1093 bytes} [Mon Jul 4 09:56:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:04.718674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984bfe5-a7ae-4df8-9d2d-cc18da91aec5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:04.759219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:04.840597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78919b30-615e-481e-98ca-fa0641a76046 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:04.861941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:04.868480 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984bfe5-a7ae-4df8-9d2d-cc18da91aec5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:04.917270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861f1e6-14f3-4106-82e7-eb497bc0fb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:04.917970 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861f1e6-14f3-4106-82e7-eb497bc0fb72 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:04.918252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861f1e6-14f3-4106-82e7-eb497bc0fb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:04.918527 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861f1e6-14f3-4106-82e7-eb497bc0fb72 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:04.919372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861f1e6-14f3-4106-82e7-eb497bc0fb72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:04.919676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861f1e6-14f3-4106-82e7-eb497bc0fb72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:04.921119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861f1e6-14f3-4106-82e7-eb497bc0fb72 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:04.926876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1eac6-3a1b-408d-ba7a-67df65aa7cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:04.927678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1eac6-3a1b-408d-ba7a-67df65aa7cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:04.927957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1eac6-3a1b-408d-ba7a-67df65aa7cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:04.928230 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1eac6-3a1b-408d-ba7a-67df65aa7cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:04.953466 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1eac6-3a1b-408d-ba7a-67df65aa7cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:04.954984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1eac6-3a1b-408d-ba7a-67df65aa7cf3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:04.960854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 309/619] 10.222.0.79 () {56 vars in 1133 bytes} [Mon Jul 4 09:56:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:04.969767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861f1e6-14f3-4106-82e7-eb497bc0fb72 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d5a1eac6-3a1b-408d-ba7a-67df65aa7cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:04.980165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:04.981241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:04.982172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:04.982172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:04.982172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:04.982172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:04.982172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 311/620] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:04.992703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1eac6-3a1b-408d-ba7a-67df65aa7cf3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:05.072966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861f1e6-14f3-4106-82e7-eb497bc0fb72 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:05.089996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:05.127836 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1eac6-3a1b-408d-ba7a-67df65aa7cf3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:05.136176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c81b5-4140-45e1-8d6e-64d7b1781949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:05.137286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c81b5-4140-45e1-8d6e-64d7b1781949 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:05.137739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c81b5-4140-45e1-8d6e-64d7b1781949 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:05.138149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c81b5-4140-45e1-8d6e-64d7b1781949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:05.197429 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c0b9f-10fc-4f87-bfed-172e844e6904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:05.198741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c0b9f-10fc-4f87-bfed-172e844e6904 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:05.199951 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c0b9f-10fc-4f87-bfed-172e844e6904 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:05.201095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c0b9f-10fc-4f87-bfed-172e844e6904 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:05.201995 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c0b9f-10fc-4f87-bfed-172e844e6904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:05.202611 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c0b9f-10fc-4f87-bfed-172e844e6904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:05.205470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c0b9f-10fc-4f87-bfed-172e844e6904 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:05.224824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 312/621] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:05.230550 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c0b9f-10fc-4f87-bfed-172e844e6904 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:05.233780 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c81b5-4140-45e1-8d6e-64d7b1781949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:05.235862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c81b5-4140-45e1-8d6e-64d7b1781949 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:05.258519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:05.338371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c0b9f-10fc-4f87-bfed-172e844e6904 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:05.374441 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9720fe-d022-49ae-8837-1e8b25a162b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:05.375865 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9720fe-d022-49ae-8837-1e8b25a162b6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:05.376370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9720fe-d022-49ae-8837-1e8b25a162b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:05.376835 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9720fe-d022-49ae-8837-1e8b25a162b6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:05.384853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9720fe-d022-49ae-8837-1e8b25a162b6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:05.386854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9720fe-d022-49ae-8837-1e8b25a162b6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-7b9720fe-d022-49ae-8837-1e8b25a162b6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:05.402146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:05.403130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:05.404074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:05.404074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:05.404074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:05.404074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:05.404074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:05.404074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:05.404074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:05.404074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:05.404074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:56:05.404074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:05.407062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 313/622] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:56:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:05.412884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9720fe-d022-49ae-8837-1e8b25a162b6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:05.417890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:05.445722 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 310/623] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:56:05.450779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c81b5-4140-45e1-8d6e-64d7b1781949 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:05.516983 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9720fe-d022-49ae-8837-1e8b25a162b6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:05.550117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:05.554208 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3848884-9abf-4ae7-bba5-16a4c2eba55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:05.555324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3848884-9abf-4ae7-bba5-16a4c2eba55b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:05.555821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3848884-9abf-4ae7-bba5-16a4c2eba55b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:05.556301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3848884-9abf-4ae7-bba5-16a4c2eba55b None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:05.557472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3848884-9abf-4ae7-bba5-16a4c2eba55b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:05.558050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3848884-9abf-4ae7-bba5-16a4c2eba55b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:05.559799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3848884-9abf-4ae7-bba5-16a4c2eba55b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:05.580042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 314/624] 10.222.0.79 () {56 vars in 1135 bytes} [Mon Jul 4 09:56:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:05.585896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3848884-9abf-4ae7-bba5-16a4c2eba55b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:05.627026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c81b5-4140-45e1-8d6e-64d7b1781949 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:05.689260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3848884-9abf-4ae7-bba5-16a4c2eba55b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:05.700912 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f8b1a-515f-4ad7-b4a3-d8bce7dae742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:05.701970 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f8b1a-515f-4ad7-b4a3-d8bce7dae742 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:05.702103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f8b1a-515f-4ad7-b4a3-d8bce7dae742 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:05.702315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f8b1a-515f-4ad7-b4a3-d8bce7dae742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:05.712857 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f8b1a-515f-4ad7-b4a3-d8bce7dae742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:05.714359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f8b1a-515f-4ad7-b4a3-d8bce7dae742 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:05.728185 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574c2da-dcc1-4ac3-bd0e-fe83f443a8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:05.729387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574c2da-dcc1-4ac3-bd0e-fe83f443a8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:05.729904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574c2da-dcc1-4ac3-bd0e-fe83f443a8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:05.730369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574c2da-dcc1-4ac3-bd0e-fe83f443a8c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-de2f8b1a-515f-4ad7-b4a3-d8bce7dae742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:05.732992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:05.733914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:05.734819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:05.734819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:05.734819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:05.734819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:05.734819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 311/625] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:05.736167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf99d1-968c-4d91-8514-6325d18a3bb8 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:05.736274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf99d1-968c-4d91-8514-6325d18a3bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:05.736473 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf99d1-968c-4d91-8514-6325d18a3bb8 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:05.741824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 312/626] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:05.742604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf99d1-968c-4d91-8514-6325d18a3bb8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:05.745813 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574c2da-dcc1-4ac3-bd0e-fe83f443a8c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:05.747653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574c2da-dcc1-4ac3-bd0e-fe83f443a8c8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:05.750369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8574c2da-dcc1-4ac3-bd0e-fe83f443a8c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:05.777732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:05.778659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:05.779614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:05.779614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:05.779614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:05.779614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:05.779614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 315/627] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:05.779614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa9196-509c-472e-83d3-6601a9840150 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:05.779614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa9196-509c-472e-83d3-6601a9840150 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:05.779614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa9196-509c-472e-83d3-6601a9840150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:05.798865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:05.803756 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a7fa9196-509c-472e-83d3-6601a9840150 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:05.805361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a7fa9196-509c-472e-83d3-6601a9840150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:05.864248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf99d1-968c-4d91-8514-6325d18a3bb8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:05.898449 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 316/628] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:05.900047 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa9196-509c-472e-83d3-6601a9840150 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:05.914356 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e925fc4-4a59-47ae-b16e-fb8b547878fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:05.915148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e925fc4-4a59-47ae-b16e-fb8b547878fa None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:05.915440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e925fc4-4a59-47ae-b16e-fb8b547878fa None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:05.915715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e925fc4-4a59-47ae-b16e-fb8b547878fa None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:05.916741 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e925fc4-4a59-47ae-b16e-fb8b547878fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:05.916741 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e925fc4-4a59-47ae-b16e-fb8b547878fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:05.918149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e925fc4-4a59-47ae-b16e-fb8b547878fa None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:05.938943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 313/629] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:05.940302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e925fc4-4a59-47ae-b16e-fb8b547878fa None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:06.018011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa9196-509c-472e-83d3-6601a9840150 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:06.045603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:06.059183 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55508dc9-e596-49ef-a379-562df4cf2e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:06.059623 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55508dc9-e596-49ef-a379-562df4cf2e27 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.059909 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55508dc9-e596-49ef-a379-562df4cf2e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.060187 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55508dc9-e596-49ef-a379-562df4cf2e27 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.061059 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55508dc9-e596-49ef-a379-562df4cf2e27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:06.061372 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55508dc9-e596-49ef-a379-562df4cf2e27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:06.062879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55508dc9-e596-49ef-a379-562df4cf2e27 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:06.066640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:06.072250 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e925fc4-4a59-47ae-b16e-fb8b547878fa None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:06.080233 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 317/630] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:06.080233 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55508dc9-e596-49ef-a379-562df4cf2e27 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:06.112225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08c27c-cf65-4112-b9b7-19d757d87c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:06.113010 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08c27c-cf65-4112-b9b7-19d757d87c5d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.113295 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08c27c-cf65-4112-b9b7-19d757d87c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.113572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08c27c-cf65-4112-b9b7-19d757d87c5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.127900 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08c27c-cf65-4112-b9b7-19d757d87c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-3d08c27c-cf65-4112-b9b7-19d757d87c5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:06.136369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:06.137752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:06.138578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:06.138578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:06.138578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:06.138578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:06.138578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:06.138578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:06.138578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:06.138578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:06.138578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:06.138578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:06.138578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:06.138578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:06.141888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 314/631] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:56:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:06.142743 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08c27c-cf65-4112-b9b7-19d757d87c5d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:06.212573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55508dc9-e596-49ef-a379-562df4cf2e27 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:06.248478 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08ad99-84c4-4831-ba5a-16ee18719aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:06.249276 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08ad99-84c4-4831-ba5a-16ee18719aec None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.249555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08ad99-84c4-4831-ba5a-16ee18719aec None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.249889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08ad99-84c4-4831-ba5a-16ee18719aec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.250943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08c27c-cf65-4112-b9b7-19d757d87c5d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:06.256944 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08ad99-84c4-4831-ba5a-16ee18719aec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:06.258552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08ad99-84c4-4831-ba5a-16ee18719aec None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-ea08ad99-84c4-4831-ba5a-16ee18719aec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:06.273048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:06.274020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:06.275100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:06.275100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:06.275100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:06.275100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:06.275100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:06.275100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:06.275100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:06.275100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:06.275100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 318/632] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:06.275100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08ad99-84c4-4831-ba5a-16ee18719aec None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:06.287565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea01f53-dfac-4497-94cc-86c8af72c9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:06.288362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea01f53-dfac-4497-94cc-86c8af72c9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.288641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea01f53-dfac-4497-94cc-86c8af72c9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.288914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea01f53-dfac-4497-94cc-86c8af72c9c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.295469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea01f53-dfac-4497-94cc-86c8af72c9c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:06.297106 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea01f53-dfac-4497-94cc-86c8af72c9c5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-1ea01f53-dfac-4497-94cc-86c8af72c9c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:06.310952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:06.311833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:06.312749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:06.312749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:06.312749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:06.312749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:06.312749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:06.312749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:06.312749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:06.312749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:06.316955 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 315/633] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:06.317344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:06.318412 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed0c95-15c5-49d9-b455-e7602cceeedb None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.318661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed0c95-15c5-49d9-b455-e7602cceeedb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.318943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed0c95-15c5-49d9-b455-e7602cceeedb None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.320496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 316/634] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:06.321554 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace44043-79e3-43f2-a958-920ad34fea1a None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.321911 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace44043-79e3-43f2-a958-920ad34fea1a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.322180 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace44043-79e3-43f2-a958-920ad34fea1a None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.323320 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 317/635] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:06.324002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace44043-79e3-43f2-a958-920ad34fea1a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:06.344551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:06.376577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08ad99-84c4-4831-ba5a-16ee18719aec None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:06.410671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74c18c-9ad2-4d78-9762-0748ea942544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:06.411342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74c18c-9ad2-4d78-9762-0748ea942544 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.411624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74c18c-9ad2-4d78-9762-0748ea942544 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.411899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74c18c-9ad2-4d78-9762-0748ea942544 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.412645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74c18c-9ad2-4d78-9762-0748ea942544 None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:06.412947 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74c18c-9ad2-4d78-9762-0748ea942544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:06.414367 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74c18c-9ad2-4d78-9762-0748ea942544 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:06.422556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 319/636] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:56:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:06.423337 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74c18c-9ad2-4d78-9762-0748ea942544 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:06.434129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace44043-79e3-43f2-a958-920ad34fea1a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:06.468933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4131e-3b94-483f-94cb-fefac7062181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:06.469767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4131e-3b94-483f-94cb-fefac7062181 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.470142 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4131e-3b94-483f-94cb-fefac7062181 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.470421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4131e-3b94-483f-94cb-fefac7062181 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.471211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4131e-3b94-483f-94cb-fefac7062181 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:06.471520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4131e-3b94-483f-94cb-fefac7062181 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:06.472972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4131e-3b94-483f-94cb-fefac7062181 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:06.484862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 318/637] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:06.486053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc9f6a-50aa-450d-a7b2-eb63608f61cb None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.486375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc9f6a-50aa-450d-a7b2-eb63608f61cb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.486600 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc9f6a-50aa-450d-a7b2-eb63608f61cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.503516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-8ffc9f6a-50aa-450d-a7b2-eb63608f61cb None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:06.504721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8ffc9f6a-50aa-450d-a7b2-eb63608f61cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:06.530841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74c18c-9ad2-4d78-9762-0748ea942544 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:06.538571 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:06.567264 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76280ca5-6271-42d0-9427-f8ec3983dfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:06.567853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76280ca5-6271-42d0-9427-f8ec3983dfd3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.568134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76280ca5-6271-42d0-9427-f8ec3983dfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.568410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76280ca5-6271-42d0-9427-f8ec3983dfd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.569198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76280ca5-6271-42d0-9427-f8ec3983dfd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:06.569514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76280ca5-6271-42d0-9427-f8ec3983dfd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:06.570997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76280ca5-6271-42d0-9427-f8ec3983dfd3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:06.578446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 319/638] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:06.578446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6d2f0-137b-42cf-83f7-ad8e4a578ef5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.578446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6d2f0-137b-42cf-83f7-ad8e4a578ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.578446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6d2f0-137b-42cf-83f7-ad8e4a578ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.583116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 320/639] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:06.583775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76280ca5-6271-42d0-9427-f8ec3983dfd3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:06.598619 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2ee6d2f0-137b-42cf-83f7-ad8e4a578ef5 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:06.599800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2ee6d2f0-137b-42cf-83f7-ad8e4a578ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:06.602215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:06.668730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 320/640] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:06.669604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6d2f0-137b-42cf-83f7-ad8e4a578ef5 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:06.703974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76280ca5-6271-42d0-9427-f8ec3983dfd3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:06.742030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53068d12-cec2-4ffa-854a-b5cbbf9e1a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:06.742832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53068d12-cec2-4ffa-854a-b5cbbf9e1a8e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.743110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53068d12-cec2-4ffa-854a-b5cbbf9e1a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.743381 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53068d12-cec2-4ffa-854a-b5cbbf9e1a8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.753883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53068d12-cec2-4ffa-854a-b5cbbf9e1a8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:06.755409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53068d12-cec2-4ffa-854a-b5cbbf9e1a8e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-53068d12-cec2-4ffa-854a-b5cbbf9e1a8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:06.774474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:06.775416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:06.776341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:06.776341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:06.776341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:06.776341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:06.776341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 321/641] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:06.776895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53068d12-cec2-4ffa-854a-b5cbbf9e1a8e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:06.782723 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6d2f0-137b-42cf-83f7-ad8e4a578ef5 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:06.805277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:06.823756 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42eae64-a882-4984-9144-6c8574c9362a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:06.824338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eae64-a882-4984-9144-6c8574c9362a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.824625 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eae64-a882-4984-9144-6c8574c9362a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.824904 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eae64-a882-4984-9144-6c8574c9362a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.826404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42eae64-a882-4984-9144-6c8574c9362a None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=ffc2409cb7fc4373a74ddfb48d501d30) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:06.826822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42eae64-a882-4984-9144-6c8574c9362a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:06.828856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42eae64-a882-4984-9144-6c8574c9362a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:06.839380 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 321/642] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:06 2022] GET /identity/v3/role_assignments?user.id=ffc2409cb7fc4373a74ddfb48d501d30&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:06.840303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42eae64-a882-4984-9144-6c8574c9362a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:06.843128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:06.908809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53068d12-cec2-4ffa-854a-b5cbbf9e1a8e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:06.953015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faede6c-5fdb-4b0b-84da-1abf3b97319f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:06.954177 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faede6c-5fdb-4b0b-84da-1abf3b97319f None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:06.954686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faede6c-5fdb-4b0b-84da-1abf3b97319f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:06.955133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faede6c-5fdb-4b0b-84da-1abf3b97319f None admin] PATH_INFO: `/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/7ec37893ec8c480990c17cb03cb0ca19/roles/fc6661dcb3994755a5c238bf37e17bb9` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:06.973166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42eae64-a882-4984-9144-6c8574c9362a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:06.973881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faede6c-5fdb-4b0b-84da-1abf3b97319f None admin] RBAC: Authorizing `identity:create_grant(project_id=f32306262e8744eab4bdc6d28e24e6d0, user_id=7ec37893ec8c480990c17cb03cb0ca19, role_id=fc6661dcb3994755a5c238bf37e17bb9)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:06.975435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faede6c-5fdb-4b0b-84da-1abf3b97319f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:07.001661 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 322/643] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:56:06 2022] PUT /identity/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/7ec37893ec8c480990c17cb03cb0ca19/roles/fc6661dcb3994755a5c238bf37e17bb9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:56:07.001661 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faede6c-5fdb-4b0b-84da-1abf3b97319f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:07.026156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b0963-df34-4ec0-ad33-f32efa7399e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:07.027334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b0963-df34-4ec0-ad33-f32efa7399e9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:07.027726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b0963-df34-4ec0-ad33-f32efa7399e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:07.028164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b0963-df34-4ec0-ad33-f32efa7399e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:07.044332 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b0963-df34-4ec0-ad33-f32efa7399e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c58b0963-df34-4ec0-ad33-f32efa7399e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:07.054450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:07.056169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 322/644] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:56:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:07.058544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b0963-df34-4ec0-ad33-f32efa7399e9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:07.111301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:07.176568 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:07.181539 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faede6c-5fdb-4b0b-84da-1abf3b97319f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:07.227490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca9e24-b0b1-49b0-b818-543a6147ff93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:07.227490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca9e24-b0b1-49b0-b818-543a6147ff93 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:07.227794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca9e24-b0b1-49b0-b818-543a6147ff93 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:07.228033 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca9e24-b0b1-49b0-b818-543a6147ff93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:07.241675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca9e24-b0b1-49b0-b818-543a6147ff93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:07.245779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b0963-df34-4ec0-ad33-f32efa7399e9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-23ca9e24-b0b1-49b0-b818-543a6147ff93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:07.249904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:07.250859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:07.251819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:07.251819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:07.251819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:07.251819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:07.251819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:07.251819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:07.251819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:07.251819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:07.251819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:07.251819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:07.251819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:07.251819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:07.251819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 323/645] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:56:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:07.252546 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca9e24-b0b1-49b0-b818-543a6147ff93 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:07.300039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b5647-96e7-408a-a488-8836ec24c2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:07.301123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b5647-96e7-408a-a488-8836ec24c2bd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:07.301652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b5647-96e7-408a-a488-8836ec24c2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:07.302120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b5647-96e7-408a-a488-8836ec24c2bd None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:07.303060 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b5647-96e7-408a-a488-8836ec24c2bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:07.303583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b5647-96e7-408a-a488-8836ec24c2bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:07.305215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b5647-96e7-408a-a488-8836ec24c2bd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:07.322491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 323/646] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:07.329604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b5647-96e7-408a-a488-8836ec24c2bd None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:07.372480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca9e24-b0b1-49b0-b818-543a6147ff93 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:07.404659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:07.416552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b31697-fc47-4564-b764-362d9d91f1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:07.417242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b31697-fc47-4564-b764-362d9d91f1be None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:07.417529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b31697-fc47-4564-b764-362d9d91f1be None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:07.417857 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b31697-fc47-4564-b764-362d9d91f1be None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:07.418616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b31697-fc47-4564-b764-362d9d91f1be None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:07.418926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b31697-fc47-4564-b764-362d9d91f1be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:07.420349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b31697-fc47-4564-b764-362d9d91f1be None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:07.433663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 324/647] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:56:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:07.433663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b31697-fc47-4564-b764-362d9d91f1be None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:07.453038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b5647-96e7-408a-a488-8836ec24c2bd None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:07.500566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:07.506584 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151f43c-3823-4ae9-82ef-3f77ab1bbbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:07.507449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151f43c-3823-4ae9-82ef-3f77ab1bbbcc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:07.507826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151f43c-3823-4ae9-82ef-3f77ab1bbbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:07.508191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151f43c-3823-4ae9-82ef-3f77ab1bbbcc None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:07.509199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151f43c-3823-4ae9-82ef-3f77ab1bbbcc None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:07.509647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151f43c-3823-4ae9-82ef-3f77ab1bbbcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:07.511282 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151f43c-3823-4ae9-82ef-3f77ab1bbbcc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:07.528253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 324/648] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:56:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:07.528253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151f43c-3823-4ae9-82ef-3f77ab1bbbcc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:07.554246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b31697-fc47-4564-b764-362d9d91f1be None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:07.592422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f001b-eaab-49d0-b4af-0e0f60c21b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:07.593363 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f001b-eaab-49d0-b4af-0e0f60c21b41 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:07.594095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f001b-eaab-49d0-b4af-0e0f60c21b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:07.594380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f001b-eaab-49d0-b4af-0e0f60c21b41 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:07.611976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f001b-eaab-49d0-b4af-0e0f60c21b41 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:07.613468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f001b-eaab-49d0-b4af-0e0f60c21b41 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-198f001b-eaab-49d0-b4af-0e0f60c21b41 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:07.629939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:07.630942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:07.631895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:07.631895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:07.631895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:07.631895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:07.631895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:07.631895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:07.631895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:07.631895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:07.631895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:56:07.631895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:07.636291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 325/649] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:56:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:07.636745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f001b-eaab-49d0-b4af-0e0f60c21b41 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:07.648761 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151f43c-3823-4ae9-82ef-3f77ab1bbbcc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:07.683273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f830f-2dbb-41ee-9ba3-e57a5d6dfd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:07.684415 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f830f-2dbb-41ee-9ba3-e57a5d6dfd63 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:07.684898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f830f-2dbb-41ee-9ba3-e57a5d6dfd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:07.685369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f830f-2dbb-41ee-9ba3-e57a5d6dfd63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:07.687948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:07.697143 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f830f-2dbb-41ee-9ba3-e57a5d6dfd63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:07.698927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f830f-2dbb-41ee-9ba3-e57a5d6dfd63 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-eb9f830f-2dbb-41ee-9ba3-e57a5d6dfd63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:07.717053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:07.718033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:07.718982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:07.718982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:07.718982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:07.718982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:07.726123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 325/650] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:07.727282 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f830f-2dbb-41ee-9ba3-e57a5d6dfd63 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:07.743205 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f001b-eaab-49d0-b4af-0e0f60c21b41 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:07.764535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:07.778432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cbd1d-35c8-4149-bb07-eead40145de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:07.779240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cbd1d-35c8-4149-bb07-eead40145de9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:07.779527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cbd1d-35c8-4149-bb07-eead40145de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:07.779803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cbd1d-35c8-4149-bb07-eead40145de9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:07.791187 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cbd1d-35c8-4149-bb07-eead40145de9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:07.792686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cbd1d-35c8-4149-bb07-eead40145de9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-db9cbd1d-35c8-4149-bb07-eead40145de9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:07.811035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:07.811991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:07.813064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:07.813064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:07.813064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:07.813064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:07.813064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 326/651] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:07.813465 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cbd1d-35c8-4149-bb07-eead40145de9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:07.846081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f830f-2dbb-41ee-9ba3-e57a5d6dfd63 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:07.882601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806943aa-6db9-4d64-b0f2-99c644faae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:07.883622 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806943aa-6db9-4d64-b0f2-99c644faae72 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:07.884102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806943aa-6db9-4d64-b0f2-99c644faae72 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:07.884605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806943aa-6db9-4d64-b0f2-99c644faae72 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:07.885680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806943aa-6db9-4d64-b0f2-99c644faae72 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:07.886216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806943aa-6db9-4d64-b0f2-99c644faae72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:07.887909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806943aa-6db9-4d64-b0f2-99c644faae72 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:07.907384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 326/652] 10.222.0.79 () {56 vars in 1137 bytes} [Mon Jul 4 09:56:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:07.908501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806943aa-6db9-4d64-b0f2-99c644faae72 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:07.922537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:07.926404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cbd1d-35c8-4149-bb07-eead40145de9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:07.962243 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1d336-d40a-433b-a8fb-6da80a2966d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:07.962910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1d336-d40a-433b-a8fb-6da80a2966d7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:07.963198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1d336-d40a-433b-a8fb-6da80a2966d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:07.963484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1d336-d40a-433b-a8fb-6da80a2966d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:07.964526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1d336-d40a-433b-a8fb-6da80a2966d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:07.964635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1d336-d40a-433b-a8fb-6da80a2966d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:07.966059 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1d336-d40a-433b-a8fb-6da80a2966d7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:07.983974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 327/653] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:07.983974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1d336-d40a-433b-a8fb-6da80a2966d7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:08.011961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:08.019992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806943aa-6db9-4d64-b0f2-99c644faae72 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:08.055515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8427bab-8d0c-428f-9fb2-243cc83954f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:08.056539 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8427bab-8d0c-428f-9fb2-243cc83954f8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:08.057021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8427bab-8d0c-428f-9fb2-243cc83954f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:08.057518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8427bab-8d0c-428f-9fb2-243cc83954f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:08.058465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8427bab-8d0c-428f-9fb2-243cc83954f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:08.058964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8427bab-8d0c-428f-9fb2-243cc83954f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:08.060543 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8427bab-8d0c-428f-9fb2-243cc83954f8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:08.072910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 327/654] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:08.074072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8427bab-8d0c-428f-9fb2-243cc83954f8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:08.090614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1d336-d40a-433b-a8fb-6da80a2966d7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:08.128535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa731ff1-0dda-4153-b0e9-bc8493fdda7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:08.129195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa731ff1-0dda-4153-b0e9-bc8493fdda7d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:08.129479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa731ff1-0dda-4153-b0e9-bc8493fdda7d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:08.129786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa731ff1-0dda-4153-b0e9-bc8493fdda7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:08.160486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:08.186513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8427bab-8d0c-428f-9fb2-243cc83954f8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:08.208339 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa731ff1-0dda-4153-b0e9-bc8493fdda7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:08.210356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa731ff1-0dda-4153-b0e9-bc8493fdda7d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:08.224701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f371f05-7e78-4dad-b1eb-7425a788c98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:08.226269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f371f05-7e78-4dad-b1eb-7425a788c98d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:08.226758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f371f05-7e78-4dad-b1eb-7425a788c98d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:08.227235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f371f05-7e78-4dad-b1eb-7425a788c98d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:08.236343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f371f05-7e78-4dad-b1eb-7425a788c98d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:08.238269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f371f05-7e78-4dad-b1eb-7425a788c98d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-4f371f05-7e78-4dad-b1eb-7425a788c98d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:08.257279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:08.258311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:08.259255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:08.259255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:08.259255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:08.259255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:08.259255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:08.259255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:08.259255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:08.259255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:08.259255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:56:08.259255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:08.266841 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 328/655] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:56:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:08.267240 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f371f05-7e78-4dad-b1eb-7425a788c98d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:08.292050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:08.337776 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 328/656] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:56:08.339843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa731ff1-0dda-4153-b0e9-bc8493fdda7d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:08.412254 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:08.437026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f371f05-7e78-4dad-b1eb-7425a788c98d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:08.473892 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa731ff1-0dda-4153-b0e9-bc8493fdda7d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:08.477636 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80b409-8e65-4574-a6de-261f44b6edd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:08.478460 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80b409-8e65-4574-a6de-261f44b6edd8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:08.478750 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80b409-8e65-4574-a6de-261f44b6edd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:08.479024 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80b409-8e65-4574-a6de-261f44b6edd8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:08.492496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80b409-8e65-4574-a6de-261f44b6edd8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:08.493849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80b409-8e65-4574-a6de-261f44b6edd8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:08.515165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acf782-1550-4570-834e-f97c3c2e7ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:08.516272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acf782-1550-4570-834e-f97c3c2e7ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:08.516771 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acf782-1550-4570-834e-f97c3c2e7ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:08.517240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acf782-1550-4570-834e-f97c3c2e7ee9 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:08.518326 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acf782-1550-4570-834e-f97c3c2e7ee9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:08.518868 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acf782-1550-4570-834e-f97c3c2e7ee9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:08.520521 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acf782-1550-4570-834e-f97c3c2e7ee9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-1d80b409-8e65-4574-a6de-261f44b6edd8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:08.523953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:08.524891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:08.526210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:08.526210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:08.526210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:08.526210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:08.526210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:08.526210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:08.526210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:08.526210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:08.526210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:56:08.526210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:08.533989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 329/657] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:56:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:08.534891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80b409-8e65-4574-a6de-261f44b6edd8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:08.548796 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 329/658] 10.222.0.79 () {56 vars in 1137 bytes} [Mon Jul 4 09:56:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:08.550108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acf782-1550-4570-834e-f97c3c2e7ee9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:08.665801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:08.680167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acf782-1550-4570-834e-f97c3c2e7ee9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:08.693793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80b409-8e65-4574-a6de-261f44b6edd8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:08.705838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:08.726165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82158e-c1fc-4f1e-ad29-7585e3d0c126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:08.727594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82158e-c1fc-4f1e-ad29-7585e3d0c126 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:08.728208 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82158e-c1fc-4f1e-ad29-7585e3d0c126 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:08.728680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82158e-c1fc-4f1e-ad29-7585e3d0c126 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:08.729867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82158e-c1fc-4f1e-ad29-7585e3d0c126 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:08.730396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82158e-c1fc-4f1e-ad29-7585e3d0c126 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:08.732052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82158e-c1fc-4f1e-ad29-7585e3d0c126 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:08.757105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 330/659] 10.222.0.79 () {56 vars in 1139 bytes} [Mon Jul 4 09:56:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:08.758663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82158e-c1fc-4f1e-ad29-7585e3d0c126 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:08.780851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2f584-fae8-41b6-a422-2986c1a45cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:08.783187 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2f584-fae8-41b6-a422-2986c1a45cac None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:08.785161 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2f584-fae8-41b6-a422-2986c1a45cac None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:08.785665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2f584-fae8-41b6-a422-2986c1a45cac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:08.800946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2f584-fae8-41b6-a422-2986c1a45cac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:08.803216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2f584-fae8-41b6-a422-2986c1a45cac None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-9ac2f584-fae8-41b6-a422-2986c1a45cac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:08.822892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:08.823861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:08.824758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:08.824758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:08.824758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:08.824758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:08.827031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 330/660] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:08.828207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2f584-fae8-41b6-a422-2986c1a45cac None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:08.986731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82158e-c1fc-4f1e-ad29-7585e3d0c126 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:08.991632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2f584-fae8-41b6-a422-2986c1a45cac None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:09.023981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:09.034408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1464f74-2032-4469-a3f6-fdddfc7b2a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:09.035457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552f4f1-a540-4660-bc46-a51fc5581d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:09.035797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1464f74-2032-4469-a3f6-fdddfc7b2a87 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:09.036177 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552f4f1-a540-4660-bc46-a51fc5581d8d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:09.036463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552f4f1-a540-4660-bc46-a51fc5581d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:09.036715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1464f74-2032-4469-a3f6-fdddfc7b2a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:09.036829 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552f4f1-a540-4660-bc46-a51fc5581d8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:09.037366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1464f74-2032-4469-a3f6-fdddfc7b2a87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:09.102139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:09.128613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552f4f1-a540-4660-bc46-a51fc5581d8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:09.130161 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552f4f1-a540-4660-bc46-a51fc5581d8d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:09.143059 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1464f74-2032-4469-a3f6-fdddfc7b2a87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:09.144722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1464f74-2032-4469-a3f6-fdddfc7b2a87 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:09.233817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 331/661] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:56:09.234809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552f4f1-a540-4660-bc46-a51fc5581d8d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:09.248918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 331/662] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:56:09.250508 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c38a46-28ae-48fd-932e-31ad72e0e0d1 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:09.251058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c38a46-28ae-48fd-932e-31ad72e0e0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:09.251489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c38a46-28ae-48fd-932e-31ad72e0e0d1 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:09.252911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 332/663] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:09.254172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43688c38-d24a-4c90-b88b-9f931e07f722 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:09.254681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43688c38-d24a-4c90-b88b-9f931e07f722 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:09.254940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43688c38-d24a-4c90-b88b-9f931e07f722 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:09.256298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 333/664] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:09.260817 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43688c38-d24a-4c90-b88b-9f931e07f722 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:09.263656 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:09.337369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552f4f1-a540-4660-bc46-a51fc5581d8d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:09.341326 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:09.367401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43688c38-d24a-4c90-b88b-9f931e07f722 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:09.372996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642409f-9c39-4f7f-8ed6-5ff238960898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:09.373769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642409f-9c39-4f7f-8ed6-5ff238960898 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:09.374075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642409f-9c39-4f7f-8ed6-5ff238960898 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:09.374356 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642409f-9c39-4f7f-8ed6-5ff238960898 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:09.375317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9642409f-9c39-4f7f-8ed6-5ff238960898 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:09.375710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9642409f-9c39-4f7f-8ed6-5ff238960898 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:09.377123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9642409f-9c39-4f7f-8ed6-5ff238960898 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:09.388757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 332/665] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:09.389556 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642409f-9c39-4f7f-8ed6-5ff238960898 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:09.405217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69519da-593b-4bfd-8b2e-e3b9ea92dd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:09.408637 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69519da-593b-4bfd-8b2e-e3b9ea92dd01 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:09.408925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69519da-593b-4bfd-8b2e-e3b9ea92dd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:09.409234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69519da-593b-4bfd-8b2e-e3b9ea92dd01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:09.419792 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69519da-593b-4bfd-8b2e-e3b9ea92dd01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:09.421289 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69519da-593b-4bfd-8b2e-e3b9ea92dd01 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b69519da-593b-4bfd-8b2e-e3b9ea92dd01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:09.439148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:09.440140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:09.441111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:09.441111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:09.441111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:09.441111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:09.441111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 334/666] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:09.441522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689e1db-fb4b-43e5-afd1-87886634c2a9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:09.441840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689e1db-fb4b-43e5-afd1-87886634c2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:09.442122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689e1db-fb4b-43e5-afd1-87886634c2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:09.461001 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1689e1db-fb4b-43e5-afd1-87886634c2a9 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:09.462246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1689e1db-fb4b-43e5-afd1-87886634c2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:09.479650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:09.494596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642409f-9c39-4f7f-8ed6-5ff238960898 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:09.529088 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd08fc-3d17-4b2e-a249-beaee8a2668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:09.529926 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd08fc-3d17-4b2e-a249-beaee8a2668c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:09.530213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd08fc-3d17-4b2e-a249-beaee8a2668c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:09.530487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd08fc-3d17-4b2e-a249-beaee8a2668c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:09.531044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 335/667] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:09.532384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65886c16-2b72-4a17-a88e-fd12b369d850 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:09.532671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65886c16-2b72-4a17-a88e-fd12b369d850 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:09.532948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65886c16-2b72-4a17-a88e-fd12b369d850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:09.550061 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd08fc-3d17-4b2e-a249-beaee8a2668c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:09.551538 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd08fc-3d17-4b2e-a249-beaee8a2668c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:09.556301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-65886c16-2b72-4a17-a88e-fd12b369d850 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:09.557488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-65886c16-2b72-4a17-a88e-fd12b369d850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-fedd08fc-3d17-4b2e-a249-beaee8a2668c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:09.569772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:09.570682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:09.571593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:09.571593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:09.571593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:09.571593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:09.571593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 333/668] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:09.571868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd08fc-3d17-4b2e-a249-beaee8a2668c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:09.615164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:09.628685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 336/669] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:09.629503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65886c16-2b72-4a17-a88e-fd12b369d850 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:09.675786 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd08fc-3d17-4b2e-a249-beaee8a2668c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:09.711989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f65f5-faed-4be8-934e-b8bf7851459c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:09.712784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f65f5-faed-4be8-934e-b8bf7851459c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:09.713063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f65f5-faed-4be8-934e-b8bf7851459c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:09.713329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f65f5-faed-4be8-934e-b8bf7851459c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:09.720293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f65f5-faed-4be8-934e-b8bf7851459c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:09.721797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f65f5-faed-4be8-934e-b8bf7851459c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:09.723587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:09.726111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65886c16-2b72-4a17-a88e-fd12b369d850 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-8c8f65f5-faed-4be8-934e-b8bf7851459c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:09.737917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:09.739017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:09.739976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:09.739976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:09.739976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:09.739976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:09.739976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:09.739976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:09.739976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:09.739976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:09.743910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 334/670] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:09.744524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f65f5-faed-4be8-934e-b8bf7851459c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:09.759362 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34afd72-996f-42a1-a47e-f0c5955fdd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:09.760027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34afd72-996f-42a1-a47e-f0c5955fdd56 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:09.760308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34afd72-996f-42a1-a47e-f0c5955fdd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:09.760581 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34afd72-996f-42a1-a47e-f0c5955fdd56 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:09.761296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34afd72-996f-42a1-a47e-f0c5955fdd56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:09.761599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34afd72-996f-42a1-a47e-f0c5955fdd56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:09.763029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34afd72-996f-42a1-a47e-f0c5955fdd56 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:09.774214 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 337/671] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:09.774990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34afd72-996f-42a1-a47e-f0c5955fdd56 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:09.822466 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:09.846488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f65f5-faed-4be8-934e-b8bf7851459c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:09.872624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34afd72-996f-42a1-a47e-f0c5955fdd56 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:09.880299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a241b-174e-4929-887d-785df93b9fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:09.881085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a241b-174e-4929-887d-785df93b9fda None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:09.881365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a241b-174e-4929-887d-785df93b9fda None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:09.881659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a241b-174e-4929-887d-785df93b9fda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:09.891745 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a241b-174e-4929-887d-785df93b9fda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-ab6a241b-174e-4929-887d-785df93b9fda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:09.899255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:09.900309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:09.901144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:09.901144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:09.901144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:09.901144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:09.901144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:09.901144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:09.901144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:09.901144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:09.901144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:09.901144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:09.901144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:09.901144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:09.901144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 335/672] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:56:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:09.901914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a241b-174e-4929-887d-785df93b9fda None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:09.910443 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7d998-ec75-4162-a2d4-00e5ac9e2a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:09.911114 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7d998-ec75-4162-a2d4-00e5ac9e2a53 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:09.911394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7d998-ec75-4162-a2d4-00e5ac9e2a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:09.911621 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7d998-ec75-4162-a2d4-00e5ac9e2a53 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:09.912346 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7d998-ec75-4162-a2d4-00e5ac9e2a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:09.912683 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7d998-ec75-4162-a2d4-00e5ac9e2a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:09.914119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7d998-ec75-4162-a2d4-00e5ac9e2a53 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:09.926167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 338/673] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:09.926942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7d998-ec75-4162-a2d4-00e5ac9e2a53 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:09.963856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:10.005397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a241b-174e-4929-887d-785df93b9fda None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:10.025962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7d998-ec75-4162-a2d4-00e5ac9e2a53 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:10.035193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:10.038948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439acaf-1347-4ac3-892e-10fd24938d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:10.039773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439acaf-1347-4ac3-892e-10fd24938d68 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:10.040033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439acaf-1347-4ac3-892e-10fd24938d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:10.040311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439acaf-1347-4ac3-892e-10fd24938d68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:10.050616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439acaf-1347-4ac3-892e-10fd24938d68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b439acaf-1347-4ac3-892e-10fd24938d68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:10.058314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:10.059236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:10.060068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:10.060068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:10.060068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:10.060068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:10.060068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:10.060068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:10.060068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:10.060068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:10.060068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:10.060068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:10.060068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:10.060068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:10.060068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 336/674] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:56:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:10.060895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff69606-ef7b-4007-90ce-322e62b273ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:10.061008 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439acaf-1347-4ac3-892e-10fd24938d68 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:10.061082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff69606-ef7b-4007-90ce-322e62b273ca None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:10.061082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff69606-ef7b-4007-90ce-322e62b273ca None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:10.061082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff69606-ef7b-4007-90ce-322e62b273ca None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:10.061883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff69606-ef7b-4007-90ce-322e62b273ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:10.062137 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff69606-ef7b-4007-90ce-322e62b273ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:10.063948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff69606-ef7b-4007-90ce-322e62b273ca None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:10.075996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 339/675] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:10.076725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff69606-ef7b-4007-90ce-322e62b273ca None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:10.172484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439acaf-1347-4ac3-892e-10fd24938d68 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:10.177629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff69606-ef7b-4007-90ce-322e62b273ca None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:10.189439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:10.206869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61107790-b491-4c47-8ee2-7254a4842061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:10.207670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61107790-b491-4c47-8ee2-7254a4842061 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:10.207956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61107790-b491-4c47-8ee2-7254a4842061 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:10.208227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61107790-b491-4c47-8ee2-7254a4842061 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:10.212289 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f6a52-b7c3-4283-bf87-132c322af028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:10.212962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f6a52-b7c3-4283-bf87-132c322af028 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:10.213244 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f6a52-b7c3-4283-bf87-132c322af028 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:10.213520 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f6a52-b7c3-4283-bf87-132c322af028 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:10.214299 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f6a52-b7c3-4283-bf87-132c322af028 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:10.214604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f6a52-b7c3-4283-bf87-132c322af028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:10.214871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61107790-b491-4c47-8ee2-7254a4842061 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:10.215990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f6a52-b7c3-4283-bf87-132c322af028 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:10.216286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61107790-b491-4c47-8ee2-7254a4842061 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:10.224190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 340/676] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:56:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:10.225058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f6a52-b7c3-4283-bf87-132c322af028 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-61107790-b491-4c47-8ee2-7254a4842061 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:10.230772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:10.231662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:10.232558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:10.232558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:10.232558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:10.232558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:10.232558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:10.232558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:10.232558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:10.232558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:10.232558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 337/677] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:10.233042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61107790-b491-4c47-8ee2-7254a4842061 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:10.256727 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:10.325099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f6a52-b7c3-4283-bf87-132c322af028 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:10.333036 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61107790-b491-4c47-8ee2-7254a4842061 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:10.359537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e609fc9-a64c-4cd3-94ec-3ae61ba62f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:10.360496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e609fc9-a64c-4cd3-94ec-3ae61ba62f91 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:10.360839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e609fc9-a64c-4cd3-94ec-3ae61ba62f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:10.361117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e609fc9-a64c-4cd3-94ec-3ae61ba62f91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:10.366319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb376b8-a767-4440-bb93-f1047a57e241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:10.366992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb376b8-a767-4440-bb93-f1047a57e241 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:10.367430 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb376b8-a767-4440-bb93-f1047a57e241 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:10.367712 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb376b8-a767-4440-bb93-f1047a57e241 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:10.368189 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e609fc9-a64c-4cd3-94ec-3ae61ba62f91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:10.368459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb376b8-a767-4440-bb93-f1047a57e241 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:10.368769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb376b8-a767-4440-bb93-f1047a57e241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:10.369668 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e609fc9-a64c-4cd3-94ec-3ae61ba62f91 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:10.370186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb376b8-a767-4440-bb93-f1047a57e241 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:10.378184 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 338/678] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:56:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:10.378970 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb376b8-a767-4440-bb93-f1047a57e241 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-6e609fc9-a64c-4cd3-94ec-3ae61ba62f91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:10.384324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:10.385352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:10.386430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:10.386430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:10.386430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:10.386430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:10.386430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:10.386430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:10.386430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:10.386430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:10.386430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 341/679] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:10.386430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e609fc9-a64c-4cd3-94ec-3ae61ba62f91 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:10.424145 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:10.476321 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:10.485173 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb376b8-a767-4440-bb93-f1047a57e241 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:10.486054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e609fc9-a64c-4cd3-94ec-3ae61ba62f91 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:10.518228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150d50e-cc00-404e-bd62-016a9a866f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:10.518906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150d50e-cc00-404e-bd62-016a9a866f96 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:10.519242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150d50e-cc00-404e-bd62-016a9a866f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:10.519519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150d50e-cc00-404e-bd62-016a9a866f96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:10.520129 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556cd6b-bdd7-483b-80f8-729aec1ce3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:10.520350 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150d50e-cc00-404e-bd62-016a9a866f96 None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=fef685a17c804e2bbfea1e07cfdf5cfe) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:10.520632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150d50e-cc00-404e-bd62-016a9a866f96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:10.521078 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556cd6b-bdd7-483b-80f8-729aec1ce3dc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:10.521424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556cd6b-bdd7-483b-80f8-729aec1ce3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:10.521807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556cd6b-bdd7-483b-80f8-729aec1ce3dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:10.522091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150d50e-cc00-404e-bd62-016a9a866f96 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:10.528133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 339/680] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:10 2022] GET /identity/v3/role_assignments?user.id=fef685a17c804e2bbfea1e07cfdf5cfe&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:10.528882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150d50e-cc00-404e-bd62-016a9a866f96 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:10.532755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556cd6b-bdd7-483b-80f8-729aec1ce3dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:10.534280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556cd6b-bdd7-483b-80f8-729aec1ce3dc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d556cd6b-bdd7-483b-80f8-729aec1ce3dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:10.555318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:10.556556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:10.558526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:10.558526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:10.558526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:10.558526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:10.558526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 342/681] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:10.560271 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556cd6b-bdd7-483b-80f8-729aec1ce3dc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:10.666736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150d50e-cc00-404e-bd62-016a9a866f96 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:10.670153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:10.679334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556cd6b-bdd7-483b-80f8-729aec1ce3dc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:10.710257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aa64a-1bd4-40c6-8d4a-133212de2095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:10.710939 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aa64a-1bd4-40c6-8d4a-133212de2095 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:10.711225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aa64a-1bd4-40c6-8d4a-133212de2095 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:10.711506 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aa64a-1bd4-40c6-8d4a-133212de2095 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:10.712315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aa64a-1bd4-40c6-8d4a-133212de2095 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:10.712630 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aa64a-1bd4-40c6-8d4a-133212de2095 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:10.714160 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aa64a-1bd4-40c6-8d4a-133212de2095 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:10.722559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01f1a5-ec3d-46bd-9019-fe0c5914bc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:10.723373 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01f1a5-ec3d-46bd-9019-fe0c5914bc96 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:10.723657 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01f1a5-ec3d-46bd-9019-fe0c5914bc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:10.723937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01f1a5-ec3d-46bd-9019-fe0c5914bc96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:10.733466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 340/682] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:10.733466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aa64a-1bd4-40c6-8d4a-133212de2095 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:10.737013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01f1a5-ec3d-46bd-9019-fe0c5914bc96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:10.738551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01f1a5-ec3d-46bd-9019-fe0c5914bc96 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:10.740947 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-7c01f1a5-ec3d-46bd-9019-fe0c5914bc96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:10.758752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:10.759697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:10.761191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:10.761191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:10.761191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:10.761191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:10.761191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 343/683] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:10.761191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01f1a5-ec3d-46bd-9019-fe0c5914bc96 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:10.842840 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aa64a-1bd4-40c6-8d4a-133212de2095 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:10.869800 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01f1a5-ec3d-46bd-9019-fe0c5914bc96 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:10.877906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03563ff0-a58a-443f-85ff-45ac2ba62ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:10.881661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03563ff0-a58a-443f-85ff-45ac2ba62ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:10.881661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03563ff0-a58a-443f-85ff-45ac2ba62ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:10.881661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03563ff0-a58a-443f-85ff-45ac2ba62ed9 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:10.881661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03563ff0-a58a-443f-85ff-45ac2ba62ed9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:10.881661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03563ff0-a58a-443f-85ff-45ac2ba62ed9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:10.881661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03563ff0-a58a-443f-85ff-45ac2ba62ed9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:10.894587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 341/684] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:10.895313 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03563ff0-a58a-443f-85ff-45ac2ba62ed9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:10.919380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b5279-d2ba-4109-b656-4ae45a067c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:10.920019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b5279-d2ba-4109-b656-4ae45a067c8e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:10.920302 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b5279-d2ba-4109-b656-4ae45a067c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:10.920622 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b5279-d2ba-4109-b656-4ae45a067c8e None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:10.921459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b5279-d2ba-4109-b656-4ae45a067c8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:10.922192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b5279-d2ba-4109-b656-4ae45a067c8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:10.922738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:10.923199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b5279-d2ba-4109-b656-4ae45a067c8e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:10.940946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 344/685] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:10.940946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b5279-d2ba-4109-b656-4ae45a067c8e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:11.000139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:11.006076 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03563ff0-a58a-443f-85ff-45ac2ba62ed9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:11.039295 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515218b0-a76c-434e-9143-f997bd9785eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:11.039948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515218b0-a76c-434e-9143-f997bd9785eb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:11.040231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515218b0-a76c-434e-9143-f997bd9785eb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:11.040511 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515218b0-a76c-434e-9143-f997bd9785eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:11.041339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515218b0-a76c-434e-9143-f997bd9785eb None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=3a1f11a4b8554a98b7626a82f01888c7) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:11.041669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515218b0-a76c-434e-9143-f997bd9785eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:11.043096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515218b0-a76c-434e-9143-f997bd9785eb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:11.055472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 342/686] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:10 2022] GET /identity/v3/role_assignments?user.id=3a1f11a4b8554a98b7626a82f01888c7&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:11.055680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b5279-d2ba-4109-b656-4ae45a067c8e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:11.056125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515218b0-a76c-434e-9143-f997bd9785eb None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:11.092219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc86c5-671b-4a91-a30d-ef4bc932b4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:11.092898 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc86c5-671b-4a91-a30d-ef4bc932b4da None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:11.093179 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc86c5-671b-4a91-a30d-ef4bc932b4da None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:11.093481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc86c5-671b-4a91-a30d-ef4bc932b4da None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:11.094461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc86c5-671b-4a91-a30d-ef4bc932b4da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:11.094766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc86c5-671b-4a91-a30d-ef4bc932b4da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:11.096297 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc86c5-671b-4a91-a30d-ef4bc932b4da None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:11.111784 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 345/687] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:11.112773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc86c5-671b-4a91-a30d-ef4bc932b4da None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:11.163542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:11.183468 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515218b0-a76c-434e-9143-f997bd9785eb None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:11.224682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03eeb7-4ef7-483b-ac3c-660e6e8fec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:11.225341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03eeb7-4ef7-483b-ac3c-660e6e8fec09 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:11.226555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03eeb7-4ef7-483b-ac3c-660e6e8fec09 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:11.226910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03eeb7-4ef7-483b-ac3c-660e6e8fec09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:11.227784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03eeb7-4ef7-483b-ac3c-660e6e8fec09 None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=c5a4af83902d4370a832ed3ea3861b56) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:11.228099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03eeb7-4ef7-483b-ac3c-660e6e8fec09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:11.229656 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03eeb7-4ef7-483b-ac3c-660e6e8fec09 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:11.245826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 343/688] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:11 2022] GET /identity/v3/role_assignments?user.id=c5a4af83902d4370a832ed3ea3861b56&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:11.246801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03eeb7-4ef7-483b-ac3c-660e6e8fec09 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:11.247671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc86c5-671b-4a91-a30d-ef4bc932b4da None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:11.264541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:11.298373 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f308bd-3ddd-474e-b89b-474518c3d291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:11.299879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f308bd-3ddd-474e-b89b-474518c3d291 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:11.300868 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f308bd-3ddd-474e-b89b-474518c3d291 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:11.301414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f308bd-3ddd-474e-b89b-474518c3d291 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:11.314796 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f308bd-3ddd-474e-b89b-474518c3d291 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:11.316605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f308bd-3ddd-474e-b89b-474518c3d291 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b1f308bd-3ddd-474e-b89b-474518c3d291 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:11.338554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:11.339806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:11.341015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:11.341015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:11.341015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:11.341015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:11.341015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:11.341015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:11.341015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:11.341015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:11.341015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:11.341015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:11.344051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 346/689] 10.222.0.79 () {56 vars in 1093 bytes} [Mon Jul 4 09:56:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:11.350203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f308bd-3ddd-474e-b89b-474518c3d291 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:11.414751 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03eeb7-4ef7-483b-ac3c-660e6e8fec09 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:11.458719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad824d-d129-497a-9799-957e879401d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:11.458978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad824d-d129-497a-9799-957e879401d8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:11.459319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad824d-d129-497a-9799-957e879401d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:11.459707 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad824d-d129-497a-9799-957e879401d8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:11.470165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad824d-d129-497a-9799-957e879401d8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:11.471481 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad824d-d129-497a-9799-957e879401d8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-41ad824d-d129-497a-9799-957e879401d8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:11.490151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:11.492236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:11.494317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:11.494317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:11.494317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:11.494317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:11.494317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:11.494317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:11.494317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:11.494317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:11.494317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:56:11.494317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:11.494317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 344/690] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:56:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:11.499258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad824d-d129-497a-9799-957e879401d8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:11.523299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:11.532157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f308bd-3ddd-474e-b89b-474518c3d291 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:11.582222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd96171-b600-49d5-ab18-d52e2735e0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:11.583876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd96171-b600-49d5-ab18-d52e2735e0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:11.584855 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd96171-b600-49d5-ab18-d52e2735e0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:11.585589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd96171-b600-49d5-ab18-d52e2735e0a5 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:11.587529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd96171-b600-49d5-ab18-d52e2735e0a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:11.588481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd96171-b600-49d5-ab18-d52e2735e0a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:11.591560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd96171-b600-49d5-ab18-d52e2735e0a5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:11.625575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 347/691] 10.222.0.79 () {56 vars in 1133 bytes} [Mon Jul 4 09:56:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:11.632489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd96171-b600-49d5-ab18-d52e2735e0a5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:11.655118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad824d-d129-497a-9799-957e879401d8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:11.674401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:11.701559 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f8191-df46-4b01-8f81-fdd3da107f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:11.703170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f8191-df46-4b01-8f81-fdd3da107f91 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:11.703672 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f8191-df46-4b01-8f81-fdd3da107f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:11.704294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f8191-df46-4b01-8f81-fdd3da107f91 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:11.705483 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f8191-df46-4b01-8f81-fdd3da107f91 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:11.706039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f8191-df46-4b01-8f81-fdd3da107f91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:11.708504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f8191-df46-4b01-8f81-fdd3da107f91 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:11.730770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 345/692] 10.222.0.79 () {56 vars in 1135 bytes} [Mon Jul 4 09:56:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:11.736250 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f8191-df46-4b01-8f81-fdd3da107f91 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:11.776392 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd96171-b600-49d5-ab18-d52e2735e0a5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:11.845869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d2b41-97f9-4bd5-8b14-e1c7207c4f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:11.847275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d2b41-97f9-4bd5-8b14-e1c7207c4f34 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:11.847562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d2b41-97f9-4bd5-8b14-e1c7207c4f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:11.847839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d2b41-97f9-4bd5-8b14-e1c7207c4f34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:11.857927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:11.917517 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f8191-df46-4b01-8f81-fdd3da107f91 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:11.928938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d2b41-97f9-4bd5-8b14-e1c7207c4f34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:11.930505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d2b41-97f9-4bd5-8b14-e1c7207c4f34 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:11.963256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:11.995814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632028c-5c16-4607-afd9-ccaeaf2f25e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:11.996492 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632028c-5c16-4607-afd9-ccaeaf2f25e2 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:11.996785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632028c-5c16-4607-afd9-ccaeaf2f25e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:11.997067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632028c-5c16-4607-afd9-ccaeaf2f25e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:12.046311 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 348/693] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:56:12.054518 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d2b41-97f9-4bd5-8b14-e1c7207c4f34 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:12.138567 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632028c-5c16-4607-afd9-ccaeaf2f25e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:12.140054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632028c-5c16-4607-afd9-ccaeaf2f25e2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:12.216289 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d2b41-97f9-4bd5-8b14-e1c7207c4f34 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:12.261471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:12.273379 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1a821-a223-471a-ad8c-4aac8daac21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:12.274690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1a821-a223-471a-ad8c-4aac8daac21e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:12.276084 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1a821-a223-471a-ad8c-4aac8daac21e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:12.277374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1a821-a223-471a-ad8c-4aac8daac21e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:12.294549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:12.304473 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1a821-a223-471a-ad8c-4aac8daac21e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:12.306530 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1a821-a223-471a-ad8c-4aac8daac21e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-84a1a821-a223-471a-ad8c-4aac8daac21e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:12.329022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:12.330193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:12.331170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:12.331170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:12.331170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:12.331170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:12.337850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 349/694] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:12.339670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1a821-a223-471a-ad8c-4aac8daac21e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:12.385701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 346/695] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:56:12.391171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632028c-5c16-4607-afd9-ccaeaf2f25e2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:12.455986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1a821-a223-471a-ad8c-4aac8daac21e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:12.497079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdf283-559a-4ff4-92c7-b34b469578fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:12.498496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdf283-559a-4ff4-92c7-b34b469578fc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:12.499228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdf283-559a-4ff4-92c7-b34b469578fc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:12.499778 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdf283-559a-4ff4-92c7-b34b469578fc None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:12.500862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fdf283-559a-4ff4-92c7-b34b469578fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:12.501375 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fdf283-559a-4ff4-92c7-b34b469578fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:12.503269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fdf283-559a-4ff4-92c7-b34b469578fc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:12.521972 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 350/696] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:12.524202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdf283-559a-4ff4-92c7-b34b469578fc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:12.578646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:12.646919 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdf283-559a-4ff4-92c7-b34b469578fc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:12.647815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632028c-5c16-4607-afd9-ccaeaf2f25e2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:12.686228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeecd04-c14d-4040-8103-93b38634324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:12.687038 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeecd04-c14d-4040-8103-93b38634324d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:12.687528 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeecd04-c14d-4040-8103-93b38634324d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:12.687815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeecd04-c14d-4040-8103-93b38634324d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:12.699767 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeecd04-c14d-4040-8103-93b38634324d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:12.702315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeecd04-c14d-4040-8103-93b38634324d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:12.708363 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f2a2d-a103-4865-8d8e-c32acd341991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:12.709169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f2a2d-a103-4865-8d8e-c32acd341991 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:12.709450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f2a2d-a103-4865-8d8e-c32acd341991 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:12.709771 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f2a2d-a103-4865-8d8e-c32acd341991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:12.720187 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f2a2d-a103-4865-8d8e-c32acd341991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:12.721648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f2a2d-a103-4865-8d8e-c32acd341991 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-cdeecd04-c14d-4040-8103-93b38634324d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:12.723414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:12.724315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:12.725222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:12.725222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:12.725222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:12.725222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:12.725222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:12.725222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:12.725222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:12.725222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:12.731565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 351/697] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:12.732863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c830a9a-f08e-4668-a300-5997bb37e06c None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:12.733143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c830a9a-f08e-4668-a300-5997bb37e06c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:12.733415 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c830a9a-f08e-4668-a300-5997bb37e06c None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:12.739164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 352/698] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:12.739957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c830a9a-f08e-4668-a300-5997bb37e06c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-056f2a2d-a103-4865-8d8e-c32acd341991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:12.747093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:12.747991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:12.748927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:12.748927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:12.748927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:12.748927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:12.751386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 347/699] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:12.753711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b125d0f-a6a7-4e95-b5c5-16b781257e65 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:12.754170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b125d0f-a6a7-4e95-b5c5-16b781257e65 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:12.754902 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b125d0f-a6a7-4e95-b5c5-16b781257e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:12.776862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2b125d0f-a6a7-4e95-b5c5-16b781257e65 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:12.779384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2b125d0f-a6a7-4e95-b5c5-16b781257e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:12.801095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:12.853185 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c830a9a-f08e-4668-a300-5997bb37e06c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:12.856709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:12.873698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 348/700] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:12.874874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b125d0f-a6a7-4e95-b5c5-16b781257e65 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:12.912351 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad95ab-5847-4dae-a033-9ef2ff5e3e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:12.913186 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad95ab-5847-4dae-a033-9ef2ff5e3e60 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:12.918267 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad95ab-5847-4dae-a033-9ef2ff5e3e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:12.918267 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad95ab-5847-4dae-a033-9ef2ff5e3e60 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:12.918267 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad95ab-5847-4dae-a033-9ef2ff5e3e60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:12.918267 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad95ab-5847-4dae-a033-9ef2ff5e3e60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:12.918267 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad95ab-5847-4dae-a033-9ef2ff5e3e60 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:12.934775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 353/701] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:12.936145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad95ab-5847-4dae-a033-9ef2ff5e3e60 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:13.013706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b125d0f-a6a7-4e95-b5c5-16b781257e65 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:13.043159 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad95ab-5847-4dae-a033-9ef2ff5e3e60 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:13.059176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed9341-1008-4f19-80a0-51f17eadb5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:13.060085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9341-1008-4f19-80a0-51f17eadb5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:13.060372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9341-1008-4f19-80a0-51f17eadb5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:13.060650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9341-1008-4f19-80a0-51f17eadb5a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:13.061777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed9341-1008-4f19-80a0-51f17eadb5a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:13.062090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed9341-1008-4f19-80a0-51f17eadb5a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:13.063685 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed9341-1008-4f19-80a0-51f17eadb5a5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:13.079227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e2f20-de1d-4d58-b805-e71040aa4229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:13.079752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 349/702] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:13.080583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed9341-1008-4f19-80a0-51f17eadb5a5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:13.084739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e2f20-de1d-4d58-b805-e71040aa4229 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:13.085127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e2f20-de1d-4d58-b805-e71040aa4229 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:13.085559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e2f20-de1d-4d58-b805-e71040aa4229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:13.097395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e2f20-de1d-4d58-b805-e71040aa4229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:13.097771 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:13.099252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e2f20-de1d-4d58-b805-e71040aa4229 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-f58e2f20-de1d-4d58-b805-e71040aa4229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:13.119768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:13.124968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:13.134130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:13.134130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:13.134130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:13.134130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:13.134130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 354/703] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:13.134130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e2f20-de1d-4d58-b805-e71040aa4229 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:13.154161 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:13.205085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed9341-1008-4f19-80a0-51f17eadb5a5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:13.247379 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e2f20-de1d-4d58-b805-e71040aa4229 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:13.248513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061d6b1-2f70-419e-9ba6-94e3f98498af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:13.249422 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061d6b1-2f70-419e-9ba6-94e3f98498af None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:13.249833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061d6b1-2f70-419e-9ba6-94e3f98498af None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:13.251057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061d6b1-2f70-419e-9ba6-94e3f98498af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:13.251899 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061d6b1-2f70-419e-9ba6-94e3f98498af None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=ffc2409cb7fc4373a74ddfb48d501d30) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:13.252210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061d6b1-2f70-419e-9ba6-94e3f98498af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:13.253660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061d6b1-2f70-419e-9ba6-94e3f98498af None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:13.261118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 350/704] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:13 2022] GET /identity/v3/role_assignments?user.id=ffc2409cb7fc4373a74ddfb48d501d30&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=a3de9df253294d069dc1867fa26600c3 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:13.262073 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061d6b1-2f70-419e-9ba6-94e3f98498af None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:13.290986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb953c8-047f-476f-a294-efc639c2e74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:13.292118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb953c8-047f-476f-a294-efc639c2e74c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:13.292838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb953c8-047f-476f-a294-efc639c2e74c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:13.293273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb953c8-047f-476f-a294-efc639c2e74c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:13.303370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb953c8-047f-476f-a294-efc639c2e74c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:13.305006 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb953c8-047f-476f-a294-efc639c2e74c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-acb953c8-047f-476f-a294-efc639c2e74c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:13.325128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:13.326324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:13.327994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:13.327994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:13.327994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:13.327994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:13.327994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:13.327994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:13.327994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:13.327994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:13.327994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 355/705] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:13.334530 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb953c8-047f-476f-a294-efc639c2e74c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:13.387892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:13.444682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061d6b1-2f70-419e-9ba6-94e3f98498af None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:13.492521 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:13.493461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40926646-b5f6-4d98-9af3-c9f23bf560b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:13.494373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40926646-b5f6-4d98-9af3-c9f23bf560b5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:13.495762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40926646-b5f6-4d98-9af3-c9f23bf560b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:13.495762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40926646-b5f6-4d98-9af3-c9f23bf560b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:13.495981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40926646-b5f6-4d98-9af3-c9f23bf560b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:13.496403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40926646-b5f6-4d98-9af3-c9f23bf560b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:13.497933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40926646-b5f6-4d98-9af3-c9f23bf560b5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:13.523624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 351/706] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:13.527584 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40926646-b5f6-4d98-9af3-c9f23bf560b5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:13.543575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb953c8-047f-476f-a294-efc639c2e74c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:13.611495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4e0b2-a780-4eba-b46b-a6c9fb37e336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:13.612171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e0b2-a780-4eba-b46b-a6c9fb37e336 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:13.612447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e0b2-a780-4eba-b46b-a6c9fb37e336 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:13.612726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e0b2-a780-4eba-b46b-a6c9fb37e336 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:13.614514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4e0b2-a780-4eba-b46b-a6c9fb37e336 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:13.614973 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4e0b2-a780-4eba-b46b-a6c9fb37e336 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:13.616414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4e0b2-a780-4eba-b46b-a6c9fb37e336 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:13.629710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 356/707] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:13.631052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b640015-0e47-47e9-bf8c-409fcd77fcdf None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:13.631344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b640015-0e47-47e9-bf8c-409fcd77fcdf None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:13.631625 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b640015-0e47-47e9-bf8c-409fcd77fcdf None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:13.632852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 357/708] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:13.637728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b640015-0e47-47e9-bf8c-409fcd77fcdf None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:13.683732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40926646-b5f6-4d98-9af3-c9f23bf560b5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:13.730452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867d530-60b2-477e-a425-8fb62b378463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:13.731705 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867d530-60b2-477e-a425-8fb62b378463 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:13.732050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867d530-60b2-477e-a425-8fb62b378463 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:13.732593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867d530-60b2-477e-a425-8fb62b378463 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:13.733307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867d530-60b2-477e-a425-8fb62b378463 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:13.735697 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867d530-60b2-477e-a425-8fb62b378463 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:13.745960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:13.752733 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b640015-0e47-47e9-bf8c-409fcd77fcdf None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:13.766439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 352/709] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:56:13 2022] POST /identity/v3/users => generated 254 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:56:13.768359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707469a-1fca-4ee2-a3c2-469fb7e6f42e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:13.769335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707469a-1fca-4ee2-a3c2-469fb7e6f42e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:13.769880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707469a-1fca-4ee2-a3c2-469fb7e6f42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:13.793790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-6707469a-1fca-4ee2-a3c2-469fb7e6f42e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:13.794723 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cf7c5-7a7c-4052-8ae3-b3c0a4700d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:13.795558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6707469a-1fca-4ee2-a3c2-469fb7e6f42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:13.795676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cf7c5-7a7c-4052-8ae3-b3c0a4700d9b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:13.795875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cf7c5-7a7c-4052-8ae3-b3c0a4700d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:13.796156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cf7c5-7a7c-4052-8ae3-b3c0a4700d9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:13.797919 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cf7c5-7a7c-4052-8ae3-b3c0a4700d9b None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=7ec37893ec8c480990c17cb03cb0ca19) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:13.798248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cf7c5-7a7c-4052-8ae3-b3c0a4700d9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:13.800600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cf7c5-7a7c-4052-8ae3-b3c0a4700d9b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:13.803997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:13.808374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 358/710] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:13 2022] GET /identity/v3/role_assignments?user.id=7ec37893ec8c480990c17cb03cb0ca19&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:13.896407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 353/711] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:13.940478 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cf7c5-7a7c-4052-8ae3-b3c0a4700d9b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:14.078590 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76450d-8089-499c-8e60-22a38e7143c8 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:14.078590 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76450d-8089-499c-8e60-22a38e7143c8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:14.078590 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76450d-8089-499c-8e60-22a38e7143c8 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:14.080344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 354/712] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:14.086173 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23778102-44ae-466f-b406-75817f581fd8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:14.086460 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23778102-44ae-466f-b406-75817f581fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:14.086744 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23778102-44ae-466f-b406-75817f581fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:14.091261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cf7c5-7a7c-4052-8ae3-b3c0a4700d9b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:14.114471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-23778102-44ae-466f-b406-75817f581fd8 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:14.115933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-23778102-44ae-466f-b406-75817f581fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:14.141288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7438f32-e9e6-44d1-a9f0-dbd4c336904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:14.142387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7438f32-e9e6-44d1-a9f0-dbd4c336904e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:14.142828 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7438f32-e9e6-44d1-a9f0-dbd4c336904e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:14.143113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7438f32-e9e6-44d1-a9f0-dbd4c336904e None admin] PATH_INFO: `/v3/services/network` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:14.143774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7438f32-e9e6-44d1-a9f0-dbd4c336904e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:14.145176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7438f32-e9e6-44d1-a9f0-dbd4c336904e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-f7438f32-e9e6-44d1-a9f0-dbd4c336904e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:14.160065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:14.161255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:14.162160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 359/713] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:13 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:14.173021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7438f32-e9e6-44d1-a9f0-dbd4c336904e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:14.207844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 355/714] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:56:14.233202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23778102-44ae-466f-b406-75817f581fd8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:14.250697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:14.293313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7438f32-e9e6-44d1-a9f0-dbd4c336904e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:14.301186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:14.355286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23778102-44ae-466f-b406-75817f581fd8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:14.391821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac10328-1937-43ba-bdee-617fb6121eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:14.393021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac10328-1937-43ba-bdee-617fb6121eae None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:14.393498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac10328-1937-43ba-bdee-617fb6121eae None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:14.394041 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac10328-1937-43ba-bdee-617fb6121eae None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:14.394981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac10328-1937-43ba-bdee-617fb6121eae None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:14.396579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac10328-1937-43ba-bdee-617fb6121eae None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:14.410352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6ae40-3279-44c3-a87b-e4eb3ba32ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:14.411048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6ae40-3279-44c3-a87b-e4eb3ba32ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:14.411338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6ae40-3279-44c3-a87b-e4eb3ba32ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:14.411624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6ae40-3279-44c3-a87b-e4eb3ba32ba1 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-4ac10328-1937-43ba-bdee-617fb6121eae None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:14.412286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:14.413214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:56:14.414094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:14.414863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6ae40-3279-44c3-a87b-e4eb3ba32ba1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:14.414863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6ae40-3279-44c3-a87b-e4eb3ba32ba1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:14.414863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6ae40-3279-44c3-a87b-e4eb3ba32ba1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:14.417341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 356/715] 10.222.0.79 () {56 vars in 1129 bytes} [Mon Jul 4 09:56:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:14.423664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac10328-1937-43ba-bdee-617fb6121eae None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:14.439931 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 360/716] 10.222.0.79 () {56 vars in 1112 bytes} [Mon Jul 4 09:56:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:14.446578 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6ae40-3279-44c3-a87b-e4eb3ba32ba1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:14.533068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac10328-1937-43ba-bdee-617fb6121eae None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:14.569013 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:14.571116 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77f86c-d7bd-4c7e-b731-19ec85f770cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:14.572042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77f86c-d7bd-4c7e-b731-19ec85f770cb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:14.572525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77f86c-d7bd-4c7e-b731-19ec85f770cb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:14.573025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77f86c-d7bd-4c7e-b731-19ec85f770cb None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:14.574144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77f86c-d7bd-4c7e-b731-19ec85f770cb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:14.578281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77f86c-d7bd-4c7e-b731-19ec85f770cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:14.578281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77f86c-d7bd-4c7e-b731-19ec85f770cb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:14.582859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 357/717] 10.222.0.79 () {56 vars in 1124 bytes} [Mon Jul 4 09:56:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:14.588551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77f86c-d7bd-4c7e-b731-19ec85f770cb None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:14.705397 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6ae40-3279-44c3-a87b-e4eb3ba32ba1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:14.713214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77f86c-d7bd-4c7e-b731-19ec85f770cb None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:14.738007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:14.747288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbf1a7-5bef-4404-9146-ec3f0c0ec044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:14.747968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbf1a7-5bef-4404-9146-ec3f0c0ec044 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:14.748258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbf1a7-5bef-4404-9146-ec3f0c0ec044 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:14.748541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbf1a7-5bef-4404-9146-ec3f0c0ec044 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:14.749253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbf1a7-5bef-4404-9146-ec3f0c0ec044 None admin] RBAC: Adding query filter params (type=network) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:14.749757 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbf1a7-5bef-4404-9146-ec3f0c0ec044 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:14.751322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbf1a7-5bef-4404-9146-ec3f0c0ec044 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:14.764085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 361/718] 10.222.0.79 () {56 vars in 1112 bytes} [Mon Jul 4 09:56:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:14.783758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf6ee5-567f-4214-9130-f6dfac60b6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:14.784454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf6ee5-567f-4214-9130-f6dfac60b6dc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:14.784733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf6ee5-567f-4214-9130-f6dfac60b6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:14.785004 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf6ee5-567f-4214-9130-f6dfac60b6dc None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:14.785749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf6ee5-567f-4214-9130-f6dfac60b6dc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:14.786057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf6ee5-567f-4214-9130-f6dfac60b6dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:14.787453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf6ee5-567f-4214-9130-f6dfac60b6dc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:14.805377 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 358/719] 10.222.0.79 () {56 vars in 1124 bytes} [Mon Jul 4 09:56:14 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:15.667237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e9755-2ef5-421d-bf51-6602cc0568ef None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:15.668282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e9755-2ef5-421d-bf51-6602cc0568ef None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:15.668765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e9755-2ef5-421d-bf51-6602cc0568ef None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:15.670642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 362/720] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:15.676444 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c18765-3b43-4911-b625-a331368a4449 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:15.677064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c18765-3b43-4911-b625-a331368a4449 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:15.677559 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c18765-3b43-4911-b625-a331368a4449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:15.698703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-84c18765-3b43-4911-b625-a331368a4449 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:15.699924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-84c18765-3b43-4911-b625-a331368a4449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:15.749285 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:15.782084 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 359/721] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:15.816924 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e9755-2ef5-421d-bf51-6602cc0568ef None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:15.967302 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e9755-2ef5-421d-bf51-6602cc0568ef None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:16.015211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93371633-7e04-4bc4-8f6a-edbea2f3d2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:16.016140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93371633-7e04-4bc4-8f6a-edbea2f3d2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:16.016498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93371633-7e04-4bc4-8f6a-edbea2f3d2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:16.016777 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93371633-7e04-4bc4-8f6a-edbea2f3d2f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:16.031474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93371633-7e04-4bc4-8f6a-edbea2f3d2f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-93371633-7e04-4bc4-8f6a-edbea2f3d2f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:16.039072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:16.040005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:16.040832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:16.040832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:16.040832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:16.040832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:16.040832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:16.040832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:16.040832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:16.040832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:16.040832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:16.040832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:16.040832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:16.040832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:16.040832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 363/722] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:56:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:16.046376 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c18765-3b43-4911-b625-a331368a4449 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:16.199284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c18765-3b43-4911-b625-a331368a4449 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:16.256300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5427a19-4fc5-46ef-be03-9ea76e0398b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:16.256905 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5427a19-4fc5-46ef-be03-9ea76e0398b5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:16.257195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5427a19-4fc5-46ef-be03-9ea76e0398b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:16.257480 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5427a19-4fc5-46ef-be03-9ea76e0398b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:16.258266 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5427a19-4fc5-46ef-be03-9ea76e0398b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:16.258575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5427a19-4fc5-46ef-be03-9ea76e0398b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:16.259975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5427a19-4fc5-46ef-be03-9ea76e0398b5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:16.273979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 360/723] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:56:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:16.279561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93371633-7e04-4bc4-8f6a-edbea2f3d2f7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:16.377716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:16.442608 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93371633-7e04-4bc4-8f6a-edbea2f3d2f7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:16.486573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01487c55-2aa2-4181-8146-52aa4a8973fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:16.487381 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01487c55-2aa2-4181-8146-52aa4a8973fa None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:16.487663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01487c55-2aa2-4181-8146-52aa4a8973fa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:16.487938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01487c55-2aa2-4181-8146-52aa4a8973fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:16.507850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01487c55-2aa2-4181-8146-52aa4a8973fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:16.509267 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01487c55-2aa2-4181-8146-52aa4a8973fa None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-01487c55-2aa2-4181-8146-52aa4a8973fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:16.532479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:16.533471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:16.534552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:16.534552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:16.534552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:16.534552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:16.534552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 364/724] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:16.539592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5427a19-4fc5-46ef-be03-9ea76e0398b5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:16.595556 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:16.651328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018a3c4-4c9a-400a-aa83-df93711fc290 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:16.651620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018a3c4-4c9a-400a-aa83-df93711fc290 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:16.651904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018a3c4-4c9a-400a-aa83-df93711fc290 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:16.653502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 365/725] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:16.659403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba30b4-47ce-430d-960d-a00f562ecf3f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:16.659694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba30b4-47ce-430d-960d-a00f562ecf3f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:16.659978 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba30b4-47ce-430d-960d-a00f562ecf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:16.678289 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a5ba30b4-47ce-430d-960d-a00f562ecf3f None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:16.678945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5427a19-4fc5-46ef-be03-9ea76e0398b5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:16.679489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a5ba30b4-47ce-430d-960d-a00f562ecf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:16.713522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d58e86-14f4-49fd-819c-4d9bcdef488f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:16.714264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d58e86-14f4-49fd-819c-4d9bcdef488f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:16.714555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d58e86-14f4-49fd-819c-4d9bcdef488f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:16.714832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d58e86-14f4-49fd-819c-4d9bcdef488f None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:16.715555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d58e86-14f4-49fd-819c-4d9bcdef488f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:16.715860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d58e86-14f4-49fd-819c-4d9bcdef488f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:16.717248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d58e86-14f4-49fd-819c-4d9bcdef488f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:16.730189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 361/726] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:16.731510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a912a8-c175-44b0-9b8d-9864852895d3 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:16.731799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a912a8-c175-44b0-9b8d-9864852895d3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:16.732075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a912a8-c175-44b0-9b8d-9864852895d3 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:16.733605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 362/727] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:16.736827 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a912a8-c175-44b0-9b8d-9864852895d3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:16.762825 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 366/728] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:56:16.769659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21498bb4-0b6f-47eb-9800-9201eeb6bc31 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:16.769659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21498bb4-0b6f-47eb-9800-9201eeb6bc31 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:16.769659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21498bb4-0b6f-47eb-9800-9201eeb6bc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:16.799461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-21498bb4-0b6f-47eb-9800-9201eeb6bc31 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:16.800833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-21498bb4-0b6f-47eb-9800-9201eeb6bc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:16.840508 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:16.862661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:16.871717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a912a8-c175-44b0-9b8d-9864852895d3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:16.899612 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 367/729] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:16.900879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21498bb4-0b6f-47eb-9800-9201eeb6bc31 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:16.910904 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fad88e-3a1e-43ba-9125-04afc4e448d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:16.912130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fad88e-3a1e-43ba-9125-04afc4e448d5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:16.912635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fad88e-3a1e-43ba-9125-04afc4e448d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:16.913084 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fad88e-3a1e-43ba-9125-04afc4e448d5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:16.922235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fad88e-3a1e-43ba-9125-04afc4e448d5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:16.923973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fad88e-3a1e-43ba-9125-04afc4e448d5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-73fad88e-3a1e-43ba-9125-04afc4e448d5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:16.947730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:16.948717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:16.949674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:16.949674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:16.949674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:16.949674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:16.949674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:16.949674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:16.949674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:16.949674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:16.949674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:16.949674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:16.952295 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 363/730] 10.222.0.79 () {56 vars in 1093 bytes} [Mon Jul 4 09:56:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:16.957947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fad88e-3a1e-43ba-9125-04afc4e448d5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:17.030557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21498bb4-0b6f-47eb-9800-9201eeb6bc31 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:17.060757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fad88e-3a1e-43ba-9125-04afc4e448d5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:17.069727 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da8e2a-1908-463e-a772-64e457630309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:17.070717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da8e2a-1908-463e-a772-64e457630309 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.071124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da8e2a-1908-463e-a772-64e457630309 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.071653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da8e2a-1908-463e-a772-64e457630309 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.083513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da8e2a-1908-463e-a772-64e457630309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-68da8e2a-1908-463e-a772-64e457630309 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:17.091863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:17.092801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:17.094224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:17.094224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:17.094224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:17.094224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:17.094224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:17.094224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:17.094224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:17.094224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:17.094224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:17.094224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:17.094224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:17.094224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:17.097696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 368/731] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:56:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:17.098726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da8e2a-1908-463e-a772-64e457630309 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:17.102573 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377d998-5232-4129-b39a-196a6d1f842a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:17.103611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377d998-5232-4129-b39a-196a6d1f842a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.104075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377d998-5232-4129-b39a-196a6d1f842a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.104525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377d998-5232-4129-b39a-196a6d1f842a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.115967 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377d998-5232-4129-b39a-196a6d1f842a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b377d998-5232-4129-b39a-196a6d1f842a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:17.126986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:17.127954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:17.128800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:17.128800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:17.128800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:17.128800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:17.128800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:17.128800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:17.128800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:17.128800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:17.128800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:17.128800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:17.128800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:17.128800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:17.130441 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:17.132255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 364/732] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:56:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:17.133529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377d998-5232-4129-b39a-196a6d1f842a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:17.158941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:17.219203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da8e2a-1908-463e-a772-64e457630309 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:17.251462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377d998-5232-4129-b39a-196a6d1f842a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:17.256572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbe4df-72af-4504-a95b-ba8c0551d0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:17.257261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbe4df-72af-4504-a95b-ba8c0551d0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.257547 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbe4df-72af-4504-a95b-ba8c0551d0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.257847 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbe4df-72af-4504-a95b-ba8c0551d0e9 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.258687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbe4df-72af-4504-a95b-ba8c0551d0e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:17.258998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbe4df-72af-4504-a95b-ba8c0551d0e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:17.260455 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbe4df-72af-4504-a95b-ba8c0551d0e9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:17.279656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 369/733] 10.222.0.79 () {56 vars in 1133 bytes} [Mon Jul 4 09:56:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:17.280530 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbe4df-72af-4504-a95b-ba8c0551d0e9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:17.290631 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca128f-5d37-484c-917c-e4c088179580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:17.291234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca128f-5d37-484c-917c-e4c088179580 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.291518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca128f-5d37-484c-917c-e4c088179580 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.291791 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca128f-5d37-484c-917c-e4c088179580 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.292542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca128f-5d37-484c-917c-e4c088179580 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:17.292847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca128f-5d37-484c-917c-e4c088179580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:17.294275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca128f-5d37-484c-917c-e4c088179580 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:17.307025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 365/734] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:56:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:17.307025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba140-0a34-4315-9556-8b20580cccaf None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.307025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba140-0a34-4315-9556-8b20580cccaf None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.307025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba140-0a34-4315-9556-8b20580cccaf None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.307025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 366/735] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:17.307025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e00979-5398-4284-a43c-c1bdbf12d3ce None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.307025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e00979-5398-4284-a43c-c1bdbf12d3ce None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.307629 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e00979-5398-4284-a43c-c1bdbf12d3ce None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.308462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 367/736] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:17.309676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e00979-5398-4284-a43c-c1bdbf12d3ce None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:17.368326 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:17.387597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbe4df-72af-4504-a95b-ba8c0551d0e9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:17.388824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:17.414736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e00979-5398-4284-a43c-c1bdbf12d3ce None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:17.428376 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6c164-b358-4568-afc5-f91f43eadb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:17.429051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6c164-b358-4568-afc5-f91f43eadb9a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.429334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6c164-b358-4568-afc5-f91f43eadb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.429611 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6c164-b358-4568-afc5-f91f43eadb9a None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.430393 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6c164-b358-4568-afc5-f91f43eadb9a None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:17.430701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6c164-b358-4568-afc5-f91f43eadb9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:17.432144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6c164-b358-4568-afc5-f91f43eadb9a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:17.440069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 370/737] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:56:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:17.440850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6c164-b358-4568-afc5-f91f43eadb9a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:17.450798 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d62cce-6c55-4e7c-b282-7d12539c3eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:17.451604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d62cce-6c55-4e7c-b282-7d12539c3eef None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.451911 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d62cce-6c55-4e7c-b282-7d12539c3eef None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.452187 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d62cce-6c55-4e7c-b282-7d12539c3eef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.464392 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d62cce-6c55-4e7c-b282-7d12539c3eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:17.465879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d62cce-6c55-4e7c-b282-7d12539c3eef None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-93d62cce-6c55-4e7c-b282-7d12539c3eef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:17.485488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:17.486588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:17.487529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:17.487529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:17.487529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:17.487529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:17.487529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 368/738] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:17.487891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa9b4b-b875-47ca-9c71-e60853eaf53c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.488214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa9b4b-b875-47ca-9c71-e60853eaf53c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.488489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa9b4b-b875-47ca-9c71-e60853eaf53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.505394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-46fa9b4b-b875-47ca-9c71-e60853eaf53c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:17.506664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-46fa9b4b-b875-47ca-9c71-e60853eaf53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:17.543110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6c164-b358-4568-afc5-f91f43eadb9a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:17.575209 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 369/739] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:17.575470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e27ab8-52c2-4c56-8a95-a4cfcd42fcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:17.576208 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e27ab8-52c2-4c56-8a95-a4cfcd42fcfa None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.576488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6220a-6cbf-4c95-b61e-4316a094a69c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.576604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e27ab8-52c2-4c56-8a95-a4cfcd42fcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.576763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e27ab8-52c2-4c56-8a95-a4cfcd42fcfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.576855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6220a-6cbf-4c95-b61e-4316a094a69c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.577047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6220a-6cbf-4c95-b61e-4316a094a69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.593536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ebc6220a-6cbf-4c95-b61e-4316a094a69c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:17.595194 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ebc6220a-6cbf-4c95-b61e-4316a094a69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:17.595301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e27ab8-52c2-4c56-8a95-a4cfcd42fcfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:17.596803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e27ab8-52c2-4c56-8a95-a4cfcd42fcfa None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:17.599909 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-27e27ab8-52c2-4c56-8a95-a4cfcd42fcfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:17.616195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:17.617199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:17.618201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:17.618201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:17.618201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:17.618201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:17.618201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 371/740] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:17.618201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e27ab8-52c2-4c56-8a95-a4cfcd42fcfa None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:17.657010 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:17.669246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 370/741] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:56:17.670069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6220a-6cbf-4c95-b61e-4316a094a69c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:17.716646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e27ab8-52c2-4c56-8a95-a4cfcd42fcfa None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:17.751417 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8c8bc-a581-48d0-bbbf-d3d3a7baf15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:17.752221 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8c8bc-a581-48d0-bbbf-d3d3a7baf15c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.752504 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8c8bc-a581-48d0-bbbf-d3d3a7baf15c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.752781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8c8bc-a581-48d0-bbbf-d3d3a7baf15c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.763474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd8c8bc-a581-48d0-bbbf-d3d3a7baf15c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:17.764933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd8c8bc-a581-48d0-bbbf-d3d3a7baf15c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:17.766631 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6220a-6cbf-4c95-b61e-4316a094a69c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-edd8c8bc-a581-48d0-bbbf-d3d3a7baf15c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:17.782694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:17.783602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:17.784494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:17.784494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:17.784494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:17.784494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:17.784494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 372/742] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:17.784796 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8c8bc-a581-48d0-bbbf-d3d3a7baf15c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:17.804885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2fad5-a1bd-4c44-9322-3ab96001fb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:17.805478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2fad5-a1bd-4c44-9322-3ab96001fb1b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.805797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2fad5-a1bd-4c44-9322-3ab96001fb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.806078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2fad5-a1bd-4c44-9322-3ab96001fb1b None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.806790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2fad5-a1bd-4c44-9322-3ab96001fb1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:17.807092 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2fad5-a1bd-4c44-9322-3ab96001fb1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:17.808498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2fad5-a1bd-4c44-9322-3ab96001fb1b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:17.820118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 371/743] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:17.821214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2fad5-a1bd-4c44-9322-3ab96001fb1b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:17.834033 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:17.871277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:17.885572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8c8bc-a581-48d0-bbbf-d3d3a7baf15c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:17.919105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a781f2-a142-4173-934c-3d29fb5a4c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:17.919765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a781f2-a142-4173-934c-3d29fb5a4c2a None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.920043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a781f2-a142-4173-934c-3d29fb5a4c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.920316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a781f2-a142-4173-934c-3d29fb5a4c2a None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.920715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2fad5-a1bd-4c44-9322-3ab96001fb1b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:17.921023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a781f2-a142-4173-934c-3d29fb5a4c2a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:17.922475 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a781f2-a142-4173-934c-3d29fb5a4c2a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:17.937507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 373/744] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:56:17 2022] POST /identity/v3/services => generated 240 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:56:17.938302 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a781f2-a142-4173-934c-3d29fb5a4c2a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:17.985033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1d6c3-8993-40fb-8a39-011263e56cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:17.985498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1d6c3-8993-40fb-8a39-011263e56cfc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:17.985812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1d6c3-8993-40fb-8a39-011263e56cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:17.986081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1d6c3-8993-40fb-8a39-011263e56cfc None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:17.986814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1d6c3-8993-40fb-8a39-011263e56cfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:17.987120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1d6c3-8993-40fb-8a39-011263e56cfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:17.988544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1d6c3-8993-40fb-8a39-011263e56cfc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:18.002027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 372/745] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:18.007335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1d6c3-8993-40fb-8a39-011263e56cfc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:18.055344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a781f2-a142-4173-934c-3d29fb5a4c2a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:18.065912 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:18.097697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d378f-4d43-4f4b-b81e-ed58339c5699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:18.098628 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d378f-4d43-4f4b-b81e-ed58339c5699 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:18.099081 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d378f-4d43-4f4b-b81e-ed58339c5699 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:18.099360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d378f-4d43-4f4b-b81e-ed58339c5699 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:18.101070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d378f-4d43-4f4b-b81e-ed58339c5699 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:18.102179 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d378f-4d43-4f4b-b81e-ed58339c5699 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:18.116860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 374/746] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:56:17 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:56:18.117695 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d378f-4d43-4f4b-b81e-ed58339c5699 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:18.124445 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1d6c3-8993-40fb-8a39-011263e56cfc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:18.142003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:18.191158 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964a9d8-ae50-4886-afb6-ab4dcb512145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:18.192969 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964a9d8-ae50-4886-afb6-ab4dcb512145 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:18.193446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964a9d8-ae50-4886-afb6-ab4dcb512145 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:18.194226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964a9d8-ae50-4886-afb6-ab4dcb512145 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:18.195230 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964a9d8-ae50-4886-afb6-ab4dcb512145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:18.196115 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964a9d8-ae50-4886-afb6-ab4dcb512145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:18.198577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964a9d8-ae50-4886-afb6-ab4dcb512145 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:18.217696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 373/747] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:18.218467 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964a9d8-ae50-4886-afb6-ab4dcb512145 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:18.254561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d378f-4d43-4f4b-b81e-ed58339c5699 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:18.303941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b3e3d-2b88-45e1-ae75-09c9e5d05134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:18.305113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b3e3d-2b88-45e1-ae75-09c9e5d05134 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:18.305777 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b3e3d-2b88-45e1-ae75-09c9e5d05134 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:18.306065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b3e3d-2b88-45e1-ae75-09c9e5d05134 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:18.313602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b3e3d-2b88-45e1-ae75-09c9e5d05134 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:18.315188 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b3e3d-2b88-45e1-ae75-09c9e5d05134 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-9f0b3e3d-2b88-45e1-ae75-09c9e5d05134 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:18.332178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:18.333120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:18.334070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:18.334070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:18.334070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:18.334070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:18.334070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:18.334070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:18.334070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:18.334070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:18.334498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964a9d8-ae50-4886-afb6-ab4dcb512145 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:18.334582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 375/748] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:18.334650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b3e3d-2b88-45e1-ae75-09c9e5d05134 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:18.373926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:18.376123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecedc6-8d41-44a0-adb5-400ffdd73607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:18.377069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecedc6-8d41-44a0-adb5-400ffdd73607 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:18.377347 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecedc6-8d41-44a0-adb5-400ffdd73607 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:18.377780 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecedc6-8d41-44a0-adb5-400ffdd73607 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:18.388049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecedc6-8d41-44a0-adb5-400ffdd73607 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:18.389589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecedc6-8d41-44a0-adb5-400ffdd73607 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-2fecedc6-8d41-44a0-adb5-400ffdd73607 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:18.406977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:18.407985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:18.409531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:18.409531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:18.409531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:18.409531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:18.409531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:18.409531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:18.409531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:18.409531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:18.409531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:56:18.409531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:18.409531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 374/749] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:56:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:18.409531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecedc6-8d41-44a0-adb5-400ffdd73607 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:18.436175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:18.464046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b3e3d-2b88-45e1-ae75-09c9e5d05134 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:18.506117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33baedf1-6d28-4303-902e-3068afdc865e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:18.507153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baedf1-6d28-4303-902e-3068afdc865e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:18.507947 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baedf1-6d28-4303-902e-3068afdc865e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:18.508229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baedf1-6d28-4303-902e-3068afdc865e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:18.517498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33baedf1-6d28-4303-902e-3068afdc865e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:18.519643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33baedf1-6d28-4303-902e-3068afdc865e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:18.527437 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecedc6-8d41-44a0-adb5-400ffdd73607 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-33baedf1-6d28-4303-902e-3068afdc865e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:18.538500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:18.539480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:18.540426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:18.540426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:18.540426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:18.540426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:18.540426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:18.540426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:18.540426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:18.540426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:18.540426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:56:18.540426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:18.540962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 376/750] 10.222.0.79 () {56 vars in 1117 bytes} [Mon Jul 4 09:56:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:18.541841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33baedf1-6d28-4303-902e-3068afdc865e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:18.566717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8156a-056b-4f01-b5bb-a2a03c63a43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:18.567296 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8156a-056b-4f01-b5bb-a2a03c63a43a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:18.567575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8156a-056b-4f01-b5bb-a2a03c63a43a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:18.568302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8156a-056b-4f01-b5bb-a2a03c63a43a None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:18.569351 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8156a-056b-4f01-b5bb-a2a03c63a43a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:18.569699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8156a-056b-4f01-b5bb-a2a03c63a43a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:18.571627 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8156a-056b-4f01-b5bb-a2a03c63a43a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:18.590047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 375/751] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:18.591256 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8156a-056b-4f01-b5bb-a2a03c63a43a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:18.636598 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:18.650991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33baedf1-6d28-4303-902e-3068afdc865e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:18.686982 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c7317-2891-44ab-b8d8-7a6dff1e79d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:18.688072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c7317-2891-44ab-b8d8-7a6dff1e79d3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:18.688638 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c7317-2891-44ab-b8d8-7a6dff1e79d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:18.689143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c7317-2891-44ab-b8d8-7a6dff1e79d3 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:18.690380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c7317-2891-44ab-b8d8-7a6dff1e79d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:18.690896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c7317-2891-44ab-b8d8-7a6dff1e79d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:18.692668 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c7317-2891-44ab-b8d8-7a6dff1e79d3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:18.698005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:18.716863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8156a-056b-4f01-b5bb-a2a03c63a43a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:18.719757 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 377/752] 10.222.0.79 () {56 vars in 1135 bytes} [Mon Jul 4 09:56:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:18.725087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c7317-2891-44ab-b8d8-7a6dff1e79d3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:18.759285 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bcbfc-d595-4072-b7a1-a5b04c5db0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:18.760089 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bcbfc-d595-4072-b7a1-a5b04c5db0fe None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:18.760434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bcbfc-d595-4072-b7a1-a5b04c5db0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:18.760714 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bcbfc-d595-4072-b7a1-a5b04c5db0fe None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:18.761613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bcbfc-d595-4072-b7a1-a5b04c5db0fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:18.762086 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bcbfc-d595-4072-b7a1-a5b04c5db0fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:18.764343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bcbfc-d595-4072-b7a1-a5b04c5db0fe None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:18.786603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 376/753] 10.222.0.79 () {56 vars in 1149 bytes} [Mon Jul 4 09:56:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:18.787737 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bcbfc-d595-4072-b7a1-a5b04c5db0fe None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:18.837313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c7317-2891-44ab-b8d8-7a6dff1e79d3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:18.877419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f11ff1-2f35-4bb0-85bc-925f1e8c0e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:18.878956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f11ff1-2f35-4bb0-85bc-925f1e8c0e2c None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:18.879336 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f11ff1-2f35-4bb0-85bc-925f1e8c0e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:18.880086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f11ff1-2f35-4bb0-85bc-925f1e8c0e2c None admin] PATH_INFO: `/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/ffc2409cb7fc4373a74ddfb48d501d30/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:18.882794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f11ff1-2f35-4bb0-85bc-925f1e8c0e2c None admin] RBAC: Authorizing `identity:create_grant(project_id=f32306262e8744eab4bdc6d28e24e6d0, user_id=ffc2409cb7fc4373a74ddfb48d501d30, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:18.884519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f11ff1-2f35-4bb0-85bc-925f1e8c0e2c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:18.907847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bcbfc-d595-4072-b7a1-a5b04c5db0fe None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:18.909885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 378/754] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:56:18 2022] PUT /identity/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/ffc2409cb7fc4373a74ddfb48d501d30/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:56:18.910984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f11ff1-2f35-4bb0-85bc-925f1e8c0e2c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:18.925759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:18.954600 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb6744-9da3-4c12-8aaf-40696ed06cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:18.955994 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb6744-9da3-4c12-8aaf-40696ed06cba None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:18.956384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb6744-9da3-4c12-8aaf-40696ed06cba None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:18.956728 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb6744-9da3-4c12-8aaf-40696ed06cba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:18.970090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:19.041119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb6744-9da3-4c12-8aaf-40696ed06cba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:19.043170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb6744-9da3-4c12-8aaf-40696ed06cba None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:19.164731 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 377/755] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:56:19.170514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb6744-9da3-4c12-8aaf-40696ed06cba None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:19.171711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f11ff1-2f35-4bb0-85bc-925f1e8c0e2c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:19.226710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07063b2-1963-44f3-b806-c984568e5bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:19.227431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07063b2-1963-44f3-b806-c984568e5bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:19.227715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07063b2-1963-44f3-b806-c984568e5bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:19.227990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07063b2-1963-44f3-b806-c984568e5bd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:19.246107 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:19.292097 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb6744-9da3-4c12-8aaf-40696ed06cba None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:19.318873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07063b2-1963-44f3-b806-c984568e5bd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:19.321126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07063b2-1963-44f3-b806-c984568e5bd7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:19.330227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3f6aa-c24c-4659-8a4e-619028a4af0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:19.331107 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3f6aa-c24c-4659-8a4e-619028a4af0c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:19.331458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3f6aa-c24c-4659-8a4e-619028a4af0c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:19.331713 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:19.331809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3f6aa-c24c-4659-8a4e-619028a4af0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:19.342684 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3f6aa-c24c-4659-8a4e-619028a4af0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:19.344457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3f6aa-c24c-4659-8a4e-619028a4af0c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-89a3f6aa-c24c-4659-8a4e-619028a4af0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:19.364157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:19.366021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:19.366021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:19.366021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:19.366021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:19.366021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 378/756] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:19.371449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3f6aa-c24c-4659-8a4e-619028a4af0c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:19.453566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 379/757] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:56:19.464492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07063b2-1963-44f3-b806-c984568e5bd7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:19.483687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:19.496023 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3f6aa-c24c-4659-8a4e-619028a4af0c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:19.537530 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31b9a3-7786-4e82-941a-59ab20d67d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:19.538928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31b9a3-7786-4e82-941a-59ab20d67d26 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:19.539218 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31b9a3-7786-4e82-941a-59ab20d67d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:19.539499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31b9a3-7786-4e82-941a-59ab20d67d26 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:19.540231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31b9a3-7786-4e82-941a-59ab20d67d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:19.540541 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31b9a3-7786-4e82-941a-59ab20d67d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:19.541963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31b9a3-7786-4e82-941a-59ab20d67d26 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:19.567422 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 379/758] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:19.572454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31b9a3-7786-4e82-941a-59ab20d67d26 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:19.669342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07063b2-1963-44f3-b806-c984568e5bd7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:19.678749 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:19.729839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8d613-926a-40dc-a95c-b986429c5706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:19.731348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8d613-926a-40dc-a95c-b986429c5706 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:19.731654 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8d613-926a-40dc-a95c-b986429c5706 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:19.731930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8d613-926a-40dc-a95c-b986429c5706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:19.752970 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31b9a3-7786-4e82-941a-59ab20d67d26 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:19.759387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8d613-926a-40dc-a95c-b986429c5706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:19.761105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8d613-926a-40dc-a95c-b986429c5706 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-aba8d613-926a-40dc-a95c-b986429c5706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:19.795782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:19.796888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:19.797950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:19.797950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:19.797950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:19.797950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:19.813087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 380/759] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:19.820094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8d613-926a-40dc-a95c-b986429c5706 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:19.850386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb89ac-5f98-446c-8813-1bda1bbc1436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:19.851565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb89ac-5f98-446c-8813-1bda1bbc1436 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:19.851857 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb89ac-5f98-446c-8813-1bda1bbc1436 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:19.852138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb89ac-5f98-446c-8813-1bda1bbc1436 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:19.860409 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb89ac-5f98-446c-8813-1bda1bbc1436 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:19.861927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb89ac-5f98-446c-8813-1bda1bbc1436 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-47cb89ac-5f98-446c-8813-1bda1bbc1436 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:19.877103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:19.878276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:19.879501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:19.879501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:19.879501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:19.879501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:19.879501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:19.879501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:19.879501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:19.879501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:19.879501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 380/760] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:19.886582 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb89ac-5f98-446c-8813-1bda1bbc1436 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:19.918577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:19.981482 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8d613-926a-40dc-a95c-b986429c5706 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:20.020891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f686921-8ea6-48fb-8ede-c5422eeb5d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:20.021640 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f686921-8ea6-48fb-8ede-c5422eeb5d5c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:20.021932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f686921-8ea6-48fb-8ede-c5422eeb5d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:20.022212 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f686921-8ea6-48fb-8ede-c5422eeb5d5c None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:20.023469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f686921-8ea6-48fb-8ede-c5422eeb5d5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:20.023790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f686921-8ea6-48fb-8ede-c5422eeb5d5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:20.025188 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f686921-8ea6-48fb-8ede-c5422eeb5d5c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:20.043009 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 381/761] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:20.046641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f686921-8ea6-48fb-8ede-c5422eeb5d5c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:20.090469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:20.132982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb89ac-5f98-446c-8813-1bda1bbc1436 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:20.208612 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a602da-7ee5-4071-962a-6adc3a2571e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:20.209299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a602da-7ee5-4071-962a-6adc3a2571e1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:20.209587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a602da-7ee5-4071-962a-6adc3a2571e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:20.211170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a602da-7ee5-4071-962a-6adc3a2571e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:20.211978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a602da-7ee5-4071-962a-6adc3a2571e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:20.212296 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a602da-7ee5-4071-962a-6adc3a2571e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:20.213820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a602da-7ee5-4071-962a-6adc3a2571e1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:20.226103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f686921-8ea6-48fb-8ede-c5422eeb5d5c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:20.227142 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 381/762] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:20.228593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc3b9c-d949-4967-bcac-bac5a8227c7f None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:20.228887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc3b9c-d949-4967-bcac-bac5a8227c7f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:20.229174 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc3b9c-d949-4967-bcac-bac5a8227c7f None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:20.230449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 382/763] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:20.233235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc3b9c-d949-4967-bcac-bac5a8227c7f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:20.268458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946e002-d3fc-42d8-a0be-3721a56b9b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:20.269272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946e002-d3fc-42d8-a0be-3721a56b9b3f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:20.269557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946e002-d3fc-42d8-a0be-3721a56b9b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:20.269868 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946e002-d3fc-42d8-a0be-3721a56b9b3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:20.276960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946e002-d3fc-42d8-a0be-3721a56b9b3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:20.278463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946e002-d3fc-42d8-a0be-3721a56b9b3f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b946e002-d3fc-42d8-a0be-3721a56b9b3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:20.294348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:20.295456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:20.296460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:20.296460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:20.296460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:20.296460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:20.296460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:20.296460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:20.296460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:20.296460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:20.296460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 382/764] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:20.297942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e143878-47db-4d85-a13a-b441c27f5457 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:20.298232 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e143878-47db-4d85-a13a-b441c27f5457 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:20.298513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e143878-47db-4d85-a13a-b441c27f5457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:20.315489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:20.319307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4e143878-47db-4d85-a13a-b441c27f5457 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:20.320525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4e143878-47db-4d85-a13a-b441c27f5457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:20.349197 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc3b9c-d949-4967-bcac-bac5a8227c7f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:20.387556 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e16b1a-3960-4b08-b55c-9d3aa3936d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:20.388651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16b1a-3960-4b08-b55c-9d3aa3936d29 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:20.389249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16b1a-3960-4b08-b55c-9d3aa3936d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:20.389773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16b1a-3960-4b08-b55c-9d3aa3936d29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:20.391136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e16b1a-3960-4b08-b55c-9d3aa3936d29 None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=7ec37893ec8c480990c17cb03cb0ca19) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:20.391690 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e16b1a-3960-4b08-b55c-9d3aa3936d29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:20.393518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e16b1a-3960-4b08-b55c-9d3aa3936d29 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:20.405960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 383/766] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:20 2022] GET /identity/v3/role_assignments?user.id=7ec37893ec8c480990c17cb03cb0ca19&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=a3de9df253294d069dc1867fa26600c3 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:20.406172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 383/766] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:20.407729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e143878-47db-4d85-a13a-b441c27f5457 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:20.425961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:20.456846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910e06c-1e12-4ed7-ba54-068caf98638a None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:20.457516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910e06c-1e12-4ed7-ba54-068caf98638a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:20.458032 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910e06c-1e12-4ed7-ba54-068caf98638a None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:20.459819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 384/767] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:20.460852 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910e06c-1e12-4ed7-ba54-068caf98638a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:20.551884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e143878-47db-4d85-a13a-b441c27f5457 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:20.574761 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910e06c-1e12-4ed7-ba54-068caf98638a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:20.590055 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d63c2-9fcf-48f0-aad8-40f6b0866c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:20.590968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d63c2-9fcf-48f0-aad8-40f6b0866c99 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:20.591080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d63c2-9fcf-48f0-aad8-40f6b0866c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:20.591296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d63c2-9fcf-48f0-aad8-40f6b0866c99 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:20.592087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d63c2-9fcf-48f0-aad8-40f6b0866c99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:20.592402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d63c2-9fcf-48f0-aad8-40f6b0866c99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:20.595319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d63c2-9fcf-48f0-aad8-40f6b0866c99 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:20.613573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 384/768] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:20.615287 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b55b74-d1f1-493e-8ce0-279a857eba68 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:20.615580 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b55b74-d1f1-493e-8ce0-279a857eba68 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:20.615861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b55b74-d1f1-493e-8ce0-279a857eba68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:20.618959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a5683-ea16-4659-bdeb-fd02d20d12ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:20.619873 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a5683-ea16-4659-bdeb-fd02d20d12ef None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:20.620256 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a5683-ea16-4659-bdeb-fd02d20d12ef None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:20.620635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a5683-ea16-4659-bdeb-fd02d20d12ef None admin] PATH_INFO: `/v3/services/network` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:20.621387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a5683-ea16-4659-bdeb-fd02d20d12ef None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:20.622909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a5683-ea16-4659-bdeb-fd02d20d12ef None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:20.629926 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:20.637868 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-87b55b74-d1f1-493e-8ce0-279a857eba68 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-6c7a5683-ea16-4659-bdeb-fd02d20d12ef None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:20.639261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:20.640557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:20.641885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 385/769] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:20 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:20.642966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a5683-ea16-4659-bdeb-fd02d20d12ef None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:20.643223 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-87b55b74-d1f1-493e-8ce0-279a857eba68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:20.714580 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:20.723090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 385/770] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:20.724250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b55b74-d1f1-493e-8ce0-279a857eba68 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:20.778654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a5683-ea16-4659-bdeb-fd02d20d12ef None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:20.814236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37737a29-01ee-468b-a668-21b3878faa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:20.814831 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37737a29-01ee-468b-a668-21b3878faa2b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:20.815112 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37737a29-01ee-468b-a668-21b3878faa2b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:20.815388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37737a29-01ee-468b-a668-21b3878faa2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:20.816221 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37737a29-01ee-468b-a668-21b3878faa2b None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=5bdf239fa2bc4f1b9e484f916c438c1a) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:20.816576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37737a29-01ee-468b-a668-21b3878faa2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:20.818101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37737a29-01ee-468b-a668-21b3878faa2b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:20.830102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 386/771] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:20 2022] GET /identity/v3/role_assignments?user.id=5bdf239fa2bc4f1b9e484f916c438c1a&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:20.830892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37737a29-01ee-468b-a668-21b3878faa2b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:20.837031 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b55b74-d1f1-493e-8ce0-279a857eba68 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:20.876395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc524a2-d318-4496-8638-7c9cfa702449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:20.877535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc524a2-d318-4496-8638-7c9cfa702449 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:20.878124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc524a2-d318-4496-8638-7c9cfa702449 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:20.878583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc524a2-d318-4496-8638-7c9cfa702449 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:20.879473 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc524a2-d318-4496-8638-7c9cfa702449 None admin] RBAC: Adding query filter params (name=network) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:20.879976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc524a2-d318-4496-8638-7c9cfa702449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:20.881720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc524a2-d318-4496-8638-7c9cfa702449 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:20.887577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 386/772] 10.222.0.79 () {56 vars in 1112 bytes} [Mon Jul 4 09:56:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:20.894148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc524a2-d318-4496-8638-7c9cfa702449 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:20.958046 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:21.019324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:21.035685 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37737a29-01ee-468b-a668-21b3878faa2b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:21.063158 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc524a2-d318-4496-8638-7c9cfa702449 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:21.087677 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcedd82-9b9e-4be1-be86-6346081d015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:21.088425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcedd82-9b9e-4be1-be86-6346081d015f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:21.088706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcedd82-9b9e-4be1-be86-6346081d015f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:21.089039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcedd82-9b9e-4be1-be86-6346081d015f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:21.089737 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcedd82-9b9e-4be1-be86-6346081d015f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:21.091181 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcedd82-9b9e-4be1-be86-6346081d015f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-5fcedd82-9b9e-4be1-be86-6346081d015f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:21.105678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:21.106666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:21.107739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 387/773] 10.222.0.79 () {56 vars in 1129 bytes} [Mon Jul 4 09:56:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:21.114800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcedd82-9b9e-4be1-be86-6346081d015f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:21.142477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0f265-f0fe-45c2-a865-0c0a3ff516fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:21.144872 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f265-f0fe-45c2-a865-0c0a3ff516fd None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:21.145345 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f265-f0fe-45c2-a865-0c0a3ff516fd None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:21.146518 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f265-f0fe-45c2-a865-0c0a3ff516fd None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:21.147428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0f265-f0fe-45c2-a865-0c0a3ff516fd None admin] RBAC: Adding query filter params (type=network) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:21.149232 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0f265-f0fe-45c2-a865-0c0a3ff516fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:21.150883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0f265-f0fe-45c2-a865-0c0a3ff516fd None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:21.169130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 387/774] 10.222.0.79 () {56 vars in 1112 bytes} [Mon Jul 4 09:56:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:21.173024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0f265-f0fe-45c2-a865-0c0a3ff516fd None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:21.309552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0f265-f0fe-45c2-a865-0c0a3ff516fd None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:21.347502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:21.348472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9576e1f4-678c-4ea7-8a3e-b1cd29036d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:21.349434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576e1f4-678c-4ea7-8a3e-b1cd29036d3b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:21.349965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576e1f4-678c-4ea7-8a3e-b1cd29036d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:21.350460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576e1f4-678c-4ea7-8a3e-b1cd29036d3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:21.351779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9576e1f4-678c-4ea7-8a3e-b1cd29036d3b None admin] RBAC: Adding query filter params (interface=public, service_id=d247958af5c14ced8c4d9d434d734fdc, region_id=RegionOne) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:21.352365 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9576e1f4-678c-4ea7-8a3e-b1cd29036d3b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:21.353994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9576e1f4-678c-4ea7-8a3e-b1cd29036d3b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:21.355622 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcedd82-9b9e-4be1-be86-6346081d015f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:21.363298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 388/775] 10.222.0.79 () {56 vars in 1251 bytes} [Mon Jul 4 09:56:21 2022] GET /identity/v3/endpoints?service_id=d247958af5c14ced8c4d9d434d734fdc&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:21.368976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9576e1f4-678c-4ea7-8a3e-b1cd29036d3b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:21.418679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:21.451009 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7e732-252f-4eb4-8429-f0a37eb2eb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:21.451714 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7e732-252f-4eb4-8429-f0a37eb2eb80 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:21.452011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7e732-252f-4eb4-8429-f0a37eb2eb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:21.452295 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7e732-252f-4eb4-8429-f0a37eb2eb80 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:21.453083 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7e732-252f-4eb4-8429-f0a37eb2eb80 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:21.453396 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7e732-252f-4eb4-8429-f0a37eb2eb80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:21.454822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7e732-252f-4eb4-8429-f0a37eb2eb80 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:21.462879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 388/776] 10.222.0.79 () {56 vars in 1124 bytes} [Mon Jul 4 09:56:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:21.467498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c01b6e-68ca-4c82-9b0e-c14e894703e0 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:21.469272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c01b6e-68ca-4c82-9b0e-c14e894703e0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:21.469272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c01b6e-68ca-4c82-9b0e-c14e894703e0 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:21.470454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 389/777] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:21.471335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c01b6e-68ca-4c82-9b0e-c14e894703e0 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:21.496193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9576e1f4-678c-4ea7-8a3e-b1cd29036d3b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:21.535853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd2902-bfa7-43b5-b877-98b2e0e1bab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:21.536856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd2902-bfa7-43b5-b877-98b2e0e1bab6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:21.537333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd2902-bfa7-43b5-b877-98b2e0e1bab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:21.538000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd2902-bfa7-43b5-b877-98b2e0e1bab6 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:21.538699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd2902-bfa7-43b5-b877-98b2e0e1bab6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:21.540094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd2902-bfa7-43b5-b877-98b2e0e1bab6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:21.546364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 389/778] 10.222.0.79 () {56 vars in 1087 bytes} [Mon Jul 4 09:56:21 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:21.547964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a230cb-cce7-47c1-9ef6-034f252e9307 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:21.548498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a230cb-cce7-47c1-9ef6-034f252e9307 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:21.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a230cb-cce7-47c1-9ef6-034f252e9307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:21.568278 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-80a230cb-cce7-47c1-9ef6-034f252e9307 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:21.570023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-80a230cb-cce7-47c1-9ef6-034f252e9307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:21.609707 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c01b6e-68ca-4c82-9b0e-c14e894703e0 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:21.631376 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:21.653242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 390/779] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:21.657074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7867162-1eda-4643-a70d-f8258b33acf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:21.657897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7867162-1eda-4643-a70d-f8258b33acf4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:21.658184 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7867162-1eda-4643-a70d-f8258b33acf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:21.658458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7867162-1eda-4643-a70d-f8258b33acf4 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:21.659234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7867162-1eda-4643-a70d-f8258b33acf4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:21.659542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7867162-1eda-4643-a70d-f8258b33acf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:21.660978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7867162-1eda-4643-a70d-f8258b33acf4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:21.667614 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 390/780] 10.222.0.79 () {56 vars in 1124 bytes} [Mon Jul 4 09:56:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:21.673609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a230cb-cce7-47c1-9ef6-034f252e9307 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:21.685646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7867162-1eda-4643-a70d-f8258b33acf4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:21.802359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:21.845270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a230cb-cce7-47c1-9ef6-034f252e9307 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:21.849969 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7867162-1eda-4643-a70d-f8258b33acf4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:21.884454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdc5cc-8efb-4be2-af46-c0c6347b30c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:21.885505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdc5cc-8efb-4be2-af46-c0c6347b30c7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:21.885952 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdc5cc-8efb-4be2-af46-c0c6347b30c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:21.886369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdc5cc-8efb-4be2-af46-c0c6347b30c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:21.887430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdc5cc-8efb-4be2-af46-c0c6347b30c7 None admin] RBAC: Adding query filter params (interface=public, service_id=b623a58ab7c54d14a798c82a45d567a6, region_id=RegionOne) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:21.887545 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a62ffb-1a48-4677-99d0-49d5f231c9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:21.888669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdc5cc-8efb-4be2-af46-c0c6347b30c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:21.888800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a62ffb-1a48-4677-99d0-49d5f231c9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:21.888800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a62ffb-1a48-4677-99d0-49d5f231c9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:21.889196 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a62ffb-1a48-4677-99d0-49d5f231c9f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:21.890827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdc5cc-8efb-4be2-af46-c0c6347b30c7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:21.896904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 391/781] 10.222.0.79 () {56 vars in 1251 bytes} [Mon Jul 4 09:56:21 2022] GET /identity/v3/endpoints?service_id=b623a58ab7c54d14a798c82a45d567a6&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:21.900417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a62ffb-1a48-4677-99d0-49d5f231c9f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:21.901930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdc5cc-8efb-4be2-af46-c0c6347b30c7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-84a62ffb-1a48-4677-99d0-49d5f231c9f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:21.908405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:21.909329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:21.910207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:21.910207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:21.910207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:21.910207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:21.910207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:21.910207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:21.910207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:21.910207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:21.910207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:21.910207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:21.910207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:21.910207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:21.910207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 391/782] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:56:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:21.915525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a62ffb-1a48-4677-99d0-49d5f231c9f7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:21.959448 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:22.015659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdc5cc-8efb-4be2-af46-c0c6347b30c7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:22.051735 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51386677-202e-4119-8192-2622433bcc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:22.052792 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51386677-202e-4119-8192-2622433bcc64 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:22.053369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51386677-202e-4119-8192-2622433bcc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:22.053891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51386677-202e-4119-8192-2622433bcc64 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:22.054782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51386677-202e-4119-8192-2622433bcc64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:22.056392 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51386677-202e-4119-8192-2622433bcc64 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:22.062604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 392/783] 10.222.0.79 () {56 vars in 1087 bytes} [Mon Jul 4 09:56:21 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:22.075633 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a62ffb-1a48-4677-99d0-49d5f231c9f7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:22.096066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:22.117388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da743c79-6b98-416e-ac62-2c90799101c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:22.118616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da743c79-6b98-416e-ac62-2c90799101c7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:22.119108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da743c79-6b98-416e-ac62-2c90799101c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:22.119582 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da743c79-6b98-416e-ac62-2c90799101c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:22.120539 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da743c79-6b98-416e-ac62-2c90799101c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:22.121050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da743c79-6b98-416e-ac62-2c90799101c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:22.128955 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da743c79-6b98-416e-ac62-2c90799101c7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:22.145438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 392/784] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:56:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:22.154860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51386677-202e-4119-8192-2622433bcc64 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:22.313758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51386677-202e-4119-8192-2622433bcc64 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:22.342049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:22.366086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a955ad-1f77-4635-b4a0-8c0043f38fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:22.366914 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a955ad-1f77-4635-b4a0-8c0043f38fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:22.367201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a955ad-1f77-4635-b4a0-8c0043f38fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:22.367485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a955ad-1f77-4635-b4a0-8c0043f38fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:22.380040 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a955ad-1f77-4635-b4a0-8c0043f38fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:22.381510 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a955ad-1f77-4635-b4a0-8c0043f38fd8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-63a955ad-1f77-4635-b4a0-8c0043f38fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:22.399152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:22.400049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:22.400957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:22.400957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:22.400957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:22.400957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:22.400957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 393/785] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:22.406610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da743c79-6b98-416e-ac62-2c90799101c7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:22.535377 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da743c79-6b98-416e-ac62-2c90799101c7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:22.590362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbf7f1-010b-41c4-9d20-d216fa924c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:22.591050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbf7f1-010b-41c4-9d20-d216fa924c3b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:22.591338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbf7f1-010b-41c4-9d20-d216fa924c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:22.591614 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbf7f1-010b-41c4-9d20-d216fa924c3b None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:22.592354 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbf7f1-010b-41c4-9d20-d216fa924c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:22.592666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbf7f1-010b-41c4-9d20-d216fa924c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:22.594520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbf7f1-010b-41c4-9d20-d216fa924c3b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:22.613495 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 393/786] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:22.619746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a955ad-1f77-4635-b4a0-8c0043f38fd8 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:22.773479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a955ad-1f77-4635-b4a0-8c0043f38fd8 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:22.779703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f56eb0-1eae-4896-afc6-7dee2917eecc None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:22.779983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f56eb0-1eae-4896-afc6-7dee2917eecc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:22.780260 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f56eb0-1eae-4896-afc6-7dee2917eecc None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:22.782383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 394/787] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:22.796087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34561b4a-11b4-4621-b67b-2edbadbf7cdb None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:22.796652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34561b4a-11b4-4621-b67b-2edbadbf7cdb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:22.796998 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34561b4a-11b4-4621-b67b-2edbadbf7cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:22.825813 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549468a3-addd-4ad4-8185-f1d8cdf739e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:22.826998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549468a3-addd-4ad4-8185-f1d8cdf739e5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:22.827497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549468a3-addd-4ad4-8185-f1d8cdf739e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:22.828125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549468a3-addd-4ad4-8185-f1d8cdf739e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:22.834711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-34561b4a-11b4-4621-b67b-2edbadbf7cdb None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:22.837032 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-34561b4a-11b4-4621-b67b-2edbadbf7cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:22.844109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549468a3-addd-4ad4-8185-f1d8cdf739e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:22.845831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549468a3-addd-4ad4-8185-f1d8cdf739e5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-549468a3-addd-4ad4-8185-f1d8cdf739e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:22.869936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:22.870964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:22.871922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:22.871922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:22.871922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:22.871922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:22.871922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:22.871922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:22.871922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:22.871922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:22.871922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:22.871922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:22.879080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 394/788] 10.222.0.79 () {56 vars in 1093 bytes} [Mon Jul 4 09:56:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:22.884778 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549468a3-addd-4ad4-8185-f1d8cdf739e5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:22.901991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:22.933132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:22.966961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 395/789] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:23.005706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34561b4a-11b4-4621-b67b-2edbadbf7cdb None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:23.051868 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549468a3-addd-4ad4-8185-f1d8cdf739e5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:23.098996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954d21-1275-427d-89b3-21f1ff7d24e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:23.099684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954d21-1275-427d-89b3-21f1ff7d24e4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:23.099983 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954d21-1275-427d-89b3-21f1ff7d24e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:23.101822 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954d21-1275-427d-89b3-21f1ff7d24e4 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:23.102684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24954d21-1275-427d-89b3-21f1ff7d24e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:23.102995 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24954d21-1275-427d-89b3-21f1ff7d24e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:23.104461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24954d21-1275-427d-89b3-21f1ff7d24e4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:23.164634 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 395/790] 10.222.0.79 () {56 vars in 1133 bytes} [Mon Jul 4 09:56:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:23.166636 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954d21-1275-427d-89b3-21f1ff7d24e4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:23.216083 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34561b4a-11b4-4621-b67b-2edbadbf7cdb None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:23.261032 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc1d5d-924b-4846-b272-df6cfcd20698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:23.261878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc1d5d-924b-4846-b272-df6cfcd20698 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:23.262159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc1d5d-924b-4846-b272-df6cfcd20698 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:23.262433 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc1d5d-924b-4846-b272-df6cfcd20698 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:23.279834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc1d5d-924b-4846-b272-df6cfcd20698 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:23.282771 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-0dfc1d5d-924b-4846-b272-df6cfcd20698 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:23.287766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:23.291466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:23.292382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:23.292382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:23.292382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:23.292382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:23.292382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:23.292382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:23.292382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:23.292382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:23.292382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:23.292382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:23.292382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:23.292382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:23.294245 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 396/791] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:56:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:23.295618 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73897b-f5ef-4e3f-80f3-1279df718ba8 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:23.295931 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73897b-f5ef-4e3f-80f3-1279df718ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:23.296322 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73897b-f5ef-4e3f-80f3-1279df718ba8 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:23.298028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 397/792] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:23.298814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73897b-f5ef-4e3f-80f3-1279df718ba8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:23.306007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954d21-1275-427d-89b3-21f1ff7d24e4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:23.332777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:23.340210 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da935430-9dd6-4f90-8b36-cb3493a28fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:23.341104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da935430-9dd6-4f90-8b36-cb3493a28fea None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:23.341525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da935430-9dd6-4f90-8b36-cb3493a28fea None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:23.341999 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da935430-9dd6-4f90-8b36-cb3493a28fea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:23.408264 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da935430-9dd6-4f90-8b36-cb3493a28fea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:23.410028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da935430-9dd6-4f90-8b36-cb3493a28fea None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:23.411769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73897b-f5ef-4e3f-80f3-1279df718ba8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:23.448552 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb75457-59b7-42f8-b7c2-c9880f3767d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:23.449322 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb75457-59b7-42f8-b7c2-c9880f3767d1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:23.449734 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb75457-59b7-42f8-b7c2-c9880f3767d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:23.450086 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb75457-59b7-42f8-b7c2-c9880f3767d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:23.450954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb75457-59b7-42f8-b7c2-c9880f3767d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:23.451331 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb75457-59b7-42f8-b7c2-c9880f3767d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:23.453152 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb75457-59b7-42f8-b7c2-c9880f3767d1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:23.469937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 398/793] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:56:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:23.471621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef021d74-8039-46e4-827a-31e1a620a2de None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:23.472166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef021d74-8039-46e4-827a-31e1a620a2de None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:23.472426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef021d74-8039-46e4-827a-31e1a620a2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:23.494227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ef021d74-8039-46e4-827a-31e1a620a2de None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:23.495615 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ef021d74-8039-46e4-827a-31e1a620a2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:23.496340 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:23.516716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 396/794] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:56:23.517480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da935430-9dd6-4f90-8b36-cb3493a28fea None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:23.577790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 399/795] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:23.578994 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef021d74-8039-46e4-827a-31e1a620a2de None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:23.615233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:23.622707 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da935430-9dd6-4f90-8b36-cb3493a28fea None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:23.661532 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d54d0-d6b2-43f2-a78b-92f3a1e584cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:23.662413 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d54d0-d6b2-43f2-a78b-92f3a1e584cc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:23.662701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d54d0-d6b2-43f2-a78b-92f3a1e584cc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:23.662978 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d54d0-d6b2-43f2-a78b-92f3a1e584cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:23.674180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d54d0-d6b2-43f2-a78b-92f3a1e584cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:23.675771 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d54d0-d6b2-43f2-a78b-92f3a1e584cc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-896d54d0-d6b2-43f2-a78b-92f3a1e584cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:23.694955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:23.695904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:23.696794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:23.696794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:23.696794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:23.696794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:23.696794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 397/796] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:23.697277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d54d0-d6b2-43f2-a78b-92f3a1e584cc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:23.710720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef021d74-8039-46e4-827a-31e1a620a2de None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:23.738249 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:23.750140 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3389d69-de65-4113-a4e7-1bbaa8121f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:23.751189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3389d69-de65-4113-a4e7-1bbaa8121f13 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:23.751620 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3389d69-de65-4113-a4e7-1bbaa8121f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:23.752113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3389d69-de65-4113-a4e7-1bbaa8121f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:23.763539 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3389d69-de65-4113-a4e7-1bbaa8121f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:23.765359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3389d69-de65-4113-a4e7-1bbaa8121f13 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c3389d69-de65-4113-a4e7-1bbaa8121f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:23.793643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:23.794576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:23.795461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:23.795461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:23.795461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:23.795461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:23.795461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 400/797] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:23.795461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3389d69-de65-4113-a4e7-1bbaa8121f13 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:23.808378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d54d0-d6b2-43f2-a78b-92f3a1e584cc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:23.847925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767bfdb-166c-43b0-8d31-9120e6014da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:23.848701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767bfdb-166c-43b0-8d31-9120e6014da1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:23.848988 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767bfdb-166c-43b0-8d31-9120e6014da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:23.849264 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767bfdb-166c-43b0-8d31-9120e6014da1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:23.860890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767bfdb-166c-43b0-8d31-9120e6014da1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-2767bfdb-166c-43b0-8d31-9120e6014da1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:23.868699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:23.869653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:23.870490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:23.870490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:23.870490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:23.870490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:23.870490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:23.870490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:23.870490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:23.870490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:23.870490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:23.870490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:23.870490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:23.870490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:23.870490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 398/798] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:56:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:23.871183 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767bfdb-166c-43b0-8d31-9120e6014da1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:23.879452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:23.919984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3389d69-de65-4113-a4e7-1bbaa8121f13 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:23.968673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ed7ed-5635-4db4-98c1-94ea813055d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:23.969349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ed7ed-5635-4db4-98c1-94ea813055d9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:23.969646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ed7ed-5635-4db4-98c1-94ea813055d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:23.970446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ed7ed-5635-4db4-98c1-94ea813055d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:23.971201 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ed7ed-5635-4db4-98c1-94ea813055d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:23.971795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ed7ed-5635-4db4-98c1-94ea813055d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:23.973516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ed7ed-5635-4db4-98c1-94ea813055d9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:23.976842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:23.985359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767bfdb-166c-43b0-8d31-9120e6014da1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:23.991943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 401/799] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:23.993460 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ed7ed-5635-4db4-98c1-94ea813055d9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:24.032624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fb7c5-fb8a-4b5d-a04a-7687e8a2321f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:24.033600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fb7c5-fb8a-4b5d-a04a-7687e8a2321f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.034058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fb7c5-fb8a-4b5d-a04a-7687e8a2321f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.034484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fb7c5-fb8a-4b5d-a04a-7687e8a2321f None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.035356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fb7c5-fb8a-4b5d-a04a-7687e8a2321f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:24.035879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fb7c5-fb8a-4b5d-a04a-7687e8a2321f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:24.037432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fb7c5-fb8a-4b5d-a04a-7687e8a2321f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:24.055154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 399/800] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:24.056310 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fb7c5-fb8a-4b5d-a04a-7687e8a2321f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:24.101842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ed7ed-5635-4db4-98c1-94ea813055d9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:24.135809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:24.142269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ca966-3881-4c36-87a1-839605bfa0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:24.142951 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ca966-3881-4c36-87a1-839605bfa0ca None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.143241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ca966-3881-4c36-87a1-839605bfa0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.143514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ca966-3881-4c36-87a1-839605bfa0ca None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.144271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ca966-3881-4c36-87a1-839605bfa0ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:24.144577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ca966-3881-4c36-87a1-839605bfa0ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:24.145991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ca966-3881-4c36-87a1-839605bfa0ca None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:24.154926 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 402/801] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:56:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:24.156166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ca966-3881-4c36-87a1-839605bfa0ca None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:24.177974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fb7c5-fb8a-4b5d-a04a-7687e8a2321f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:24.215930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380b6b1-cfea-4a77-b93e-77dcb440654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:24.216859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380b6b1-cfea-4a77-b93e-77dcb440654b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.217139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380b6b1-cfea-4a77-b93e-77dcb440654b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.217577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380b6b1-cfea-4a77-b93e-77dcb440654b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.225855 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380b6b1-cfea-4a77-b93e-77dcb440654b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:24.227330 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380b6b1-cfea-4a77-b93e-77dcb440654b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-e380b6b1-cfea-4a77-b93e-77dcb440654b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:24.242146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:24.243304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:24.244837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:24.244837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:24.244837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:24.244837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:24.244837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:24.244837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:24.244837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:24.244837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:24.244837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:56:24.244837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:24.244837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 400/802] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:56:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:24.245456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdeed5-0561-4f39-962b-4cd08592c92f None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.245456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdeed5-0561-4f39-962b-4cd08592c92f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.245456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdeed5-0561-4f39-962b-4cd08592c92f None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.246866 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 401/803] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:24.251019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdeed5-0561-4f39-962b-4cd08592c92f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:24.267575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ca966-3881-4c36-87a1-839605bfa0ca None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:24.268419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:24.302651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93c59d-dd7f-4f81-b318-1957d10fb2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:24.303448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93c59d-dd7f-4f81-b318-1957d10fb2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.303726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93c59d-dd7f-4f81-b318-1957d10fb2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.304005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93c59d-dd7f-4f81-b318-1957d10fb2a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.310854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93c59d-dd7f-4f81-b318-1957d10fb2a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:24.312348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93c59d-dd7f-4f81-b318-1957d10fb2a5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-7f93c59d-dd7f-4f81-b318-1957d10fb2a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:24.327253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:24.328448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:24.329374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:24.329374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:24.329374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:24.329374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:24.329374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:24.329374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:24.329374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:24.329374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:24.329374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 403/804] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:24.329374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93c59d-dd7f-4f81-b318-1957d10fb2a5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:24.366218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdeed5-0561-4f39-962b-4cd08592c92f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:24.389846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:24.403214 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d4bae-58fd-46c2-9815-0f61018c001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:24.404020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4bae-58fd-46c2-9815-0f61018c001b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.404334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4bae-58fd-46c2-9815-0f61018c001b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.404575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4bae-58fd-46c2-9815-0f61018c001b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.416412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d4bae-58fd-46c2-9815-0f61018c001b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:24.417923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d4bae-58fd-46c2-9815-0f61018c001b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:24.435010 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93c59d-dd7f-4f81-b318-1957d10fb2a5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-a71d4bae-58fd-46c2-9815-0f61018c001b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:24.437198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:24.438327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:24.439220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:24.439220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:24.439220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:24.439220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:24.439220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 402/805] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:24.440686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2c80f-d24e-4b56-bf4e-886570a11370 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.440686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2c80f-d24e-4b56-bf4e-886570a11370 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.440686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2c80f-d24e-4b56-bf4e-886570a11370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.457868 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-13f2c80f-d24e-4b56-bf4e-886570a11370 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:24.459067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-13f2c80f-d24e-4b56-bf4e-886570a11370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:24.476874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33ef28-0252-41c2-b037-3567567f919d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:24.477549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33ef28-0252-41c2-b037-3567567f919d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.477878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33ef28-0252-41c2-b037-3567567f919d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.478165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33ef28-0252-41c2-b037-3567567f919d None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.479003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33ef28-0252-41c2-b037-3567567f919d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:24.479312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33ef28-0252-41c2-b037-3567567f919d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:24.480760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33ef28-0252-41c2-b037-3567567f919d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:24.498742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 404/806] 10.222.0.79 () {56 vars in 1135 bytes} [Mon Jul 4 09:56:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:24.499963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33ef28-0252-41c2-b037-3567567f919d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:24.524760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:24.534553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 403/807] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:24.535229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2c80f-d24e-4b56-bf4e-886570a11370 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:24.612404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33ef28-0252-41c2-b037-3567567f919d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:24.625504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:24.645719 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2c80f-d24e-4b56-bf4e-886570a11370 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:24.646250 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa68116-829c-46f4-92e3-4aff68bb19e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:24.646913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa68116-829c-46f4-92e3-4aff68bb19e1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.647199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa68116-829c-46f4-92e3-4aff68bb19e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.647475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa68116-829c-46f4-92e3-4aff68bb19e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.648265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa68116-829c-46f4-92e3-4aff68bb19e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:24.648569 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa68116-829c-46f4-92e3-4aff68bb19e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:24.650036 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa68116-829c-46f4-92e3-4aff68bb19e1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:24.661499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 405/808] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:24.662318 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa68116-829c-46f4-92e3-4aff68bb19e1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:24.681561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41731531-27c2-4360-b68b-decea30b885f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:24.682245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41731531-27c2-4360-b68b-decea30b885f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.682545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41731531-27c2-4360-b68b-decea30b885f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.682815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41731531-27c2-4360-b68b-decea30b885f None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.683529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41731531-27c2-4360-b68b-decea30b885f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:24.683832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41731531-27c2-4360-b68b-decea30b885f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:24.685222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41731531-27c2-4360-b68b-decea30b885f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:24.696851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 404/809] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:24.697839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41731531-27c2-4360-b68b-decea30b885f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:24.753411 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:24.767217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa68116-829c-46f4-92e3-4aff68bb19e1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:24.797996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41731531-27c2-4360-b68b-decea30b885f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:24.800806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4a12d-cf20-4b68-8417-5bbae58a621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:24.801610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4a12d-cf20-4b68-8417-5bbae58a621c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.801923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4a12d-cf20-4b68-8417-5bbae58a621c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.802204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4a12d-cf20-4b68-8417-5bbae58a621c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.813095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4a12d-cf20-4b68-8417-5bbae58a621c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:24.814591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4a12d-cf20-4b68-8417-5bbae58a621c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:24.831156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b662c06-331f-4868-8607-b439c477c4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:24.831948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b662c06-331f-4868-8607-b439c477c4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.832225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b662c06-331f-4868-8607-b439c477c4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-62d4a12d-cf20-4b68-8417-5bbae58a621c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:24.832387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:24.833338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:24.834278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:24.834278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:24.834278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:24.834278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:24.834495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b662c06-331f-4868-8607-b439c477c4a6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.834495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b662c06-331f-4868-8607-b439c477c4a6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:24.834628 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 406/810] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:24.834697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b662c06-331f-4868-8607-b439c477c4a6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:24.835256 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216f14b-2b64-49bb-9754-595a436e13fe None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.835461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216f14b-2b64-49bb-9754-595a436e13fe None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.835759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216f14b-2b64-49bb-9754-595a436e13fe None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.837656 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 407/811] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:24.837777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216f14b-2b64-49bb-9754-595a436e13fe None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-2b662c06-331f-4868-8607-b439c477c4a6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:24.851296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:24.852645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:24.853489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 405/812] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:24 2022] GET /identity/v3/services/network => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:24.854304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b662c06-331f-4868-8607-b439c477c4a6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:24.868741 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:24.943402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216f14b-2b64-49bb-9754-595a436e13fe None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:24.951092 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b662c06-331f-4868-8607-b439c477c4a6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:24.970331 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:24.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb76ed6-3af5-49ba-979c-b01c181082e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:24.978091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb76ed6-3af5-49ba-979c-b01c181082e4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.978374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb76ed6-3af5-49ba-979c-b01c181082e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.978652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb76ed6-3af5-49ba-979c-b01c181082e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.979478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb76ed6-3af5-49ba-979c-b01c181082e4 None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=ffc2409cb7fc4373a74ddfb48d501d30) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:24.979789 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb76ed6-3af5-49ba-979c-b01c181082e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:24.981231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb76ed6-3af5-49ba-979c-b01c181082e4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:24.985094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128277b2-075e-45ae-adad-67fbde2cf159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:24.985952 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128277b2-075e-45ae-adad-67fbde2cf159 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.986241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128277b2-075e-45ae-adad-67fbde2cf159 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.986519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128277b2-075e-45ae-adad-67fbde2cf159 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.997900 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 408/813] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:24 2022] GET /identity/v3/role_assignments?user.id=ffc2409cb7fc4373a74ddfb48d501d30&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=a3de9df253294d069dc1867fa26600c3 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:24.997900 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32075004-bf3f-4ba1-9cce-5b7ec0457a3f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:24.997900 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32075004-bf3f-4ba1-9cce-5b7ec0457a3f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:24.997900 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32075004-bf3f-4ba1-9cce-5b7ec0457a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:24.998360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128277b2-075e-45ae-adad-67fbde2cf159 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:24.998360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128277b2-075e-45ae-adad-67fbde2cf159 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:25.006594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-32075004-bf3f-4ba1-9cce-5b7ec0457a3f None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:25.007862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-32075004-bf3f-4ba1-9cce-5b7ec0457a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-128277b2-075e-45ae-adad-67fbde2cf159 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:25.012616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:25.013669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:25.014615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:25.014615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:25.014615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:25.014615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:25.014615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:25.014615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:25.014615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:25.014615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:25.014615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:56:25.014615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:25.014615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 406/814] 10.222.0.79 () {56 vars in 1117 bytes} [Mon Jul 4 09:56:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:25.015935 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128277b2-075e-45ae-adad-67fbde2cf159 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:25.093345 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 409/815] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:25.094739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32075004-bf3f-4ba1-9cce-5b7ec0457a3f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:25.130891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:25.148373 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128277b2-075e-45ae-adad-67fbde2cf159 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:25.187516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fb188-fd01-4413-80f4-371cce7f02dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:25.188190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fb188-fd01-4413-80f4-371cce7f02dd None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:25.188992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fb188-fd01-4413-80f4-371cce7f02dd None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:25.188992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fb188-fd01-4413-80f4-371cce7f02dd None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:25.189452 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fb188-fd01-4413-80f4-371cce7f02dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:25.189853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fb188-fd01-4413-80f4-371cce7f02dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:25.191268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fb188-fd01-4413-80f4-371cce7f02dd None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:25.209966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 407/816] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:25.209966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fb188-fd01-4413-80f4-371cce7f02dd None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:25.231578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32075004-bf3f-4ba1-9cce-5b7ec0457a3f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:25.253594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:25.268583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ee4fe-5936-4a05-929f-2d9929ea16ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:25.269470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ee4fe-5936-4a05-929f-2d9929ea16ec None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:25.269789 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ee4fe-5936-4a05-929f-2d9929ea16ec None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:25.270106 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ee4fe-5936-4a05-929f-2d9929ea16ec None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:25.270775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ee4fe-5936-4a05-929f-2d9929ea16ec None admin] RBAC: Adding query filter params (name=network) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:25.271082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ee4fe-5936-4a05-929f-2d9929ea16ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:25.272684 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ee4fe-5936-4a05-929f-2d9929ea16ec None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:25.282037 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 410/817] 10.222.0.79 () {56 vars in 1112 bytes} [Mon Jul 4 09:56:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:25.282037 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ee4fe-5936-4a05-929f-2d9929ea16ec None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:25.316039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fb188-fd01-4413-80f4-371cce7f02dd None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:25.366736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:25.390009 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ee4fe-5936-4a05-929f-2d9929ea16ec None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:25.428348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fca235-87d4-4ed1-be4a-a71ca0b55846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:25.429179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fca235-87d4-4ed1-be4a-a71ca0b55846 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:25.429461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fca235-87d4-4ed1-be4a-a71ca0b55846 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:25.429765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fca235-87d4-4ed1-be4a-a71ca0b55846 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:25.430432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fca235-87d4-4ed1-be4a-a71ca0b55846 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:25.431821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fca235-87d4-4ed1-be4a-a71ca0b55846 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-11fca235-87d4-4ed1-be4a-a71ca0b55846 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:25.445447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:25.446469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:25.447287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 411/818] 10.222.0.79 () {56 vars in 1129 bytes} [Mon Jul 4 09:56:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:25.448212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fca235-87d4-4ed1-be4a-a71ca0b55846 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:25.476357 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c10d93-55b4-4089-ae52-6c809bb60df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:25.476954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c10d93-55b4-4089-ae52-6c809bb60df1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:25.477233 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c10d93-55b4-4089-ae52-6c809bb60df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:25.477505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c10d93-55b4-4089-ae52-6c809bb60df1 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:25.478373 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c10d93-55b4-4089-ae52-6c809bb60df1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:25.478706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c10d93-55b4-4089-ae52-6c809bb60df1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:25.480169 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c10d93-55b4-4089-ae52-6c809bb60df1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:25.504744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 408/819] 10.222.0.79 () {56 vars in 1149 bytes} [Mon Jul 4 09:56:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:25.504744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c10d93-55b4-4089-ae52-6c809bb60df1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:25.569328 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fca235-87d4-4ed1-be4a-a71ca0b55846 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:25.604051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:25.607965 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32299d2-8710-4e11-9ddf-2da70242fec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:25.608141 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c10d93-55b4-4089-ae52-6c809bb60df1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:25.609030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32299d2-8710-4e11-9ddf-2da70242fec2 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:25.609307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32299d2-8710-4e11-9ddf-2da70242fec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:25.609581 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32299d2-8710-4e11-9ddf-2da70242fec2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:25.617195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32299d2-8710-4e11-9ddf-2da70242fec2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:25.618801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32299d2-8710-4e11-9ddf-2da70242fec2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c32299d2-8710-4e11-9ddf-2da70242fec2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:25.633849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:25.634716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:25.635601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:25.635601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:25.635601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:25.635601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:25.635601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:25.635601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:25.635601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:25.635601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:25.635601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 412/820] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:25.636081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32299d2-8710-4e11-9ddf-2da70242fec2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:25.645262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:25.650172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c2edd-cc79-4b94-8a35-e09071e219b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:25.650832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c2edd-cc79-4b94-8a35-e09071e219b0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:25.651126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c2edd-cc79-4b94-8a35-e09071e219b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:25.651393 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c2edd-cc79-4b94-8a35-e09071e219b0 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:25.652102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c2edd-cc79-4b94-8a35-e09071e219b0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:25.652409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c2edd-cc79-4b94-8a35-e09071e219b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:25.653831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c2edd-cc79-4b94-8a35-e09071e219b0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:25.658975 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 409/821] 10.222.0.79 () {56 vars in 1112 bytes} [Mon Jul 4 09:56:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:25.659764 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c2edd-cc79-4b94-8a35-e09071e219b0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:25.756041 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32299d2-8710-4e11-9ddf-2da70242fec2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:25.766822 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c2edd-cc79-4b94-8a35-e09071e219b0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:25.793457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f741e6-9ccd-4241-8b42-e10a5fcb9ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:25.794181 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f741e6-9ccd-4241-8b42-e10a5fcb9ace None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:25.794461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f741e6-9ccd-4241-8b42-e10a5fcb9ace None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:25.794733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f741e6-9ccd-4241-8b42-e10a5fcb9ace None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:25.795431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f741e6-9ccd-4241-8b42-e10a5fcb9ace None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:25.795731 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f741e6-9ccd-4241-8b42-e10a5fcb9ace None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:25.797223 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f741e6-9ccd-4241-8b42-e10a5fcb9ace None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:25.802181 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 413/822] 10.222.0.79 () {56 vars in 1124 bytes} [Mon Jul 4 09:56:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:25.802789 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f741e6-9ccd-4241-8b42-e10a5fcb9ace None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:25.807787 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c513a-4fa4-4643-a4e9-e4abb775a8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:25.808591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c513a-4fa4-4643-a4e9-e4abb775a8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:25.808878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c513a-4fa4-4643-a4e9-e4abb775a8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:25.809152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c513a-4fa4-4643-a4e9-e4abb775a8a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:25.820250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c513a-4fa4-4643-a4e9-e4abb775a8a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:25.821879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c513a-4fa4-4643-a4e9-e4abb775a8a6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-3c9c513a-4fa4-4643-a4e9-e4abb775a8a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:25.839662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:25.840559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:25.846810 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:25.846810 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:25.846810 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:25.846810 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:25.846810 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 410/823] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:25.846810 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c513a-4fa4-4643-a4e9-e4abb775a8a6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:25.863153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:25.878830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:25.924653 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f741e6-9ccd-4241-8b42-e10a5fcb9ace None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:25.964548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c513a-4fa4-4643-a4e9-e4abb775a8a6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:25.966056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d436891-869e-42d4-85e2-f3f289a997a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:25.966735 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d436891-869e-42d4-85e2-f3f289a997a8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:25.967018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d436891-869e-42d4-85e2-f3f289a997a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:25.967295 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d436891-869e-42d4-85e2-f3f289a997a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:25.968232 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d436891-869e-42d4-85e2-f3f289a997a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:25.968770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d436891-869e-42d4-85e2-f3f289a997a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:25.970267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d436891-869e-42d4-85e2-f3f289a997a8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:25.983514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 414/824] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:25.984521 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d436891-869e-42d4-85e2-f3f289a997a8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:25.998952 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf68d2-3b5f-4c11-a4b4-abb7e52dc59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:26.000076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf68d2-3b5f-4c11-a4b4-abb7e52dc59f None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:26.000715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf68d2-3b5f-4c11-a4b4-abb7e52dc59f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:26.001216 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf68d2-3b5f-4c11-a4b4-abb7e52dc59f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:26.002173 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf68d2-3b5f-4c11-a4b4-abb7e52dc59f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:26.003624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf68d2-3b5f-4c11-a4b4-abb7e52dc59f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:26.037818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 411/825] 10.222.0.79 () {60 vars in 1144 bytes} [Mon Jul 4 09:56:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:56:26.037818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf68d2-3b5f-4c11-a4b4-abb7e52dc59f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:26.097164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d436891-869e-42d4-85e2-f3f289a997a8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:26.137196 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:26.145513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:26.156688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2159a-39fe-41d8-813a-c20ea28eaf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:26.157509 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2159a-39fe-41d8-813a-c20ea28eaf92 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:26.157842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2159a-39fe-41d8-813a-c20ea28eaf92 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:26.158118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2159a-39fe-41d8-813a-c20ea28eaf92 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:26.158852 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2159a-39fe-41d8-813a-c20ea28eaf92 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:26.159208 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2159a-39fe-41d8-813a-c20ea28eaf92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:26.160823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2159a-39fe-41d8-813a-c20ea28eaf92 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:26.167874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 415/826] 10.222.0.79 () {56 vars in 1124 bytes} [Mon Jul 4 09:56:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:26.168447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2159a-39fe-41d8-813a-c20ea28eaf92 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:26.192098 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf68d2-3b5f-4c11-a4b4-abb7e52dc59f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:26.232351 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9431bae-62ed-4ca4-ad41-acde747a8089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:26.234064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9431bae-62ed-4ca4-ad41-acde747a8089 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:26.234346 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9431bae-62ed-4ca4-ad41-acde747a8089 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:26.234619 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9431bae-62ed-4ca4-ad41-acde747a8089 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:26.235541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9431bae-62ed-4ca4-ad41-acde747a8089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:26.235717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9431bae-62ed-4ca4-ad41-acde747a8089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:26.237077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9431bae-62ed-4ca4-ad41-acde747a8089 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:26.251174 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 412/827] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:26.251857 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9431bae-62ed-4ca4-ad41-acde747a8089 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:26.309530 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2159a-39fe-41d8-813a-c20ea28eaf92 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:26.348009 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e4aa5-61c7-4b9e-82c6-5430818b9bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:26.348900 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e4aa5-61c7-4b9e-82c6-5430818b9bf2 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:26.349177 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e4aa5-61c7-4b9e-82c6-5430818b9bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:26.349450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e4aa5-61c7-4b9e-82c6-5430818b9bf2 None admin] PATH_INFO: `/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/7ec37893ec8c480990c17cb03cb0ca19/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:26.351738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e4aa5-61c7-4b9e-82c6-5430818b9bf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f32306262e8744eab4bdc6d28e24e6d0, user_id=7ec37893ec8c480990c17cb03cb0ca19, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:26.353167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e4aa5-61c7-4b9e-82c6-5430818b9bf2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:26.365904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9431bae-62ed-4ca4-ad41-acde747a8089 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:26.377660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 416/828] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:56:26 2022] PUT /identity/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/7ec37893ec8c480990c17cb03cb0ca19/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:56:26.377660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e4aa5-61c7-4b9e-82c6-5430818b9bf2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:26.399078 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:26.406423 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587bae0-fe73-4ec8-8cf7-dfd75cb827f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:26.407198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587bae0-fe73-4ec8-8cf7-dfd75cb827f7 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:26.407513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587bae0-fe73-4ec8-8cf7-dfd75cb827f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:26.407862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587bae0-fe73-4ec8-8cf7-dfd75cb827f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:26.408773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587bae0-fe73-4ec8-8cf7-dfd75cb827f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:26.410261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587bae0-fe73-4ec8-8cf7-dfd75cb827f7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:26.438796 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 413/829] 10.222.0.79 () {60 vars in 1144 bytes} [Mon Jul 4 09:56:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:56:26.458115 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:26.518712 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e4aa5-61c7-4b9e-82c6-5430818b9bf2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:26.567760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b72938-6a91-444b-a862-f6e3a269fdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:26.568560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b72938-6a91-444b-a862-f6e3a269fdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:26.568849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b72938-6a91-444b-a862-f6e3a269fdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:26.569123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b72938-6a91-444b-a862-f6e3a269fdc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:26.577128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b72938-6a91-444b-a862-f6e3a269fdc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:26.578880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b72938-6a91-444b-a862-f6e3a269fdc8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c0b72938-6a91-444b-a862-f6e3a269fdc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:26.595054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:26.596074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:26.596945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:26.596945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:26.596945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:26.596945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:26.596945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:26.596945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:26.596945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:26.596945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:26.596945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 417/830] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:26.603898 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587bae0-fe73-4ec8-8cf7-dfd75cb827f7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:26.715566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587bae0-fe73-4ec8-8cf7-dfd75cb827f7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:26.750729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db3401-ad2d-4394-9b51-7762d92f7e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:26.751399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db3401-ad2d-4394-9b51-7762d92f7e8a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:26.751677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db3401-ad2d-4394-9b51-7762d92f7e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:26.751950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db3401-ad2d-4394-9b51-7762d92f7e8a None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:26.752737 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db3401-ad2d-4394-9b51-7762d92f7e8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:26.753042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db3401-ad2d-4394-9b51-7762d92f7e8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:26.754522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db3401-ad2d-4394-9b51-7762d92f7e8a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:26.767499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 414/831] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:26.773404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b72938-6a91-444b-a862-f6e3a269fdc8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:26.888664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b72938-6a91-444b-a862-f6e3a269fdc8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:26.896619 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:26.924553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28992b34-c5ec-4615-bd77-d208ad780fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:26.925447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28992b34-c5ec-4615-bd77-d208ad780fc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:26.925760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28992b34-c5ec-4615-bd77-d208ad780fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:26.926039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28992b34-c5ec-4615-bd77-d208ad780fc9 None admin] PATH_INFO: `/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/5bdf239fa2bc4f1b9e484f916c438c1a/roles/fc6661dcb3994755a5c238bf37e17bb9` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:26.940484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28992b34-c5ec-4615-bd77-d208ad780fc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f32306262e8744eab4bdc6d28e24e6d0, user_id=5bdf239fa2bc4f1b9e484f916c438c1a, role_id=fc6661dcb3994755a5c238bf37e17bb9)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:26.941983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28992b34-c5ec-4615-bd77-d208ad780fc9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:26.960831 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 418/832] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:56:26 2022] PUT /identity/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/5bdf239fa2bc4f1b9e484f916c438c1a/roles/fc6661dcb3994755a5c238bf37e17bb9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:56:27.146893 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd300813-1003-4c3a-bb29-0e80dc2224a9 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:27.147746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd300813-1003-4c3a-bb29-0e80dc2224a9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:27.148186 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd300813-1003-4c3a-bb29-0e80dc2224a9 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:27.150077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 415/833] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:27.155472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db875cc0-1fcc-4414-b991-45afeb2601c5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:27.155884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db875cc0-1fcc-4414-b991-45afeb2601c5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:27.156266 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db875cc0-1fcc-4414-b991-45afeb2601c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:27.174366 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-db875cc0-1fcc-4414-b991-45afeb2601c5 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:27.175567 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-db875cc0-1fcc-4414-b991-45afeb2601c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:27.246056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 419/834] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:27.269502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd300813-1003-4c3a-bb29-0e80dc2224a9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:27.321550 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:27.379850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd300813-1003-4c3a-bb29-0e80dc2224a9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:27.414719 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf046ea8-a06a-4b7e-9352-5d250346992e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:27.415468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf046ea8-a06a-4b7e-9352-5d250346992e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:27.415754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf046ea8-a06a-4b7e-9352-5d250346992e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:27.416032 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf046ea8-a06a-4b7e-9352-5d250346992e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:27.416680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf046ea8-a06a-4b7e-9352-5d250346992e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:27.418104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf046ea8-a06a-4b7e-9352-5d250346992e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-bf046ea8-a06a-4b7e-9352-5d250346992e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:27.430945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:27.431920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:27.433164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 416/835] 10.222.0.79 () {56 vars in 1132 bytes} [Mon Jul 4 09:56:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:27.438979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db875cc0-1fcc-4414-b991-45afeb2601c5 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:27.530649 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:27.550720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db875cc0-1fcc-4414-b991-45afeb2601c5 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:27.588307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92037d0-4531-472b-850a-d64fdf6ab0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:27.589411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92037d0-4531-472b-850a-d64fdf6ab0cc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:27.589925 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92037d0-4531-472b-850a-d64fdf6ab0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:27.590407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92037d0-4531-472b-850a-d64fdf6ab0cc None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:27.591335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92037d0-4531-472b-850a-d64fdf6ab0cc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:27.591855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92037d0-4531-472b-850a-d64fdf6ab0cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:27.593801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92037d0-4531-472b-850a-d64fdf6ab0cc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:27.599251 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 420/836] 10.222.0.79 () {56 vars in 1126 bytes} [Mon Jul 4 09:56:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:27.607609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf046ea8-a06a-4b7e-9352-5d250346992e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:27.715148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf046ea8-a06a-4b7e-9352-5d250346992e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:27.726633 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:27.751853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3eecd-8868-4c2d-a342-a958f5466ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:27.752532 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3eecd-8868-4c2d-a342-a958f5466ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:27.752821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3eecd-8868-4c2d-a342-a958f5466ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:27.753102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3eecd-8868-4c2d-a342-a958f5466ef3 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:27.753844 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3eecd-8868-4c2d-a342-a958f5466ef3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:27.754153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3eecd-8868-4c2d-a342-a958f5466ef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:27.755565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3eecd-8868-4c2d-a342-a958f5466ef3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:27.760548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 417/837] 10.222.0.79 () {56 vars in 1126 bytes} [Mon Jul 4 09:56:27 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:28.535129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f628fd-b494-406c-a7c7-41c9fcb60983 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:28.535678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f628fd-b494-406c-a7c7-41c9fcb60983 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:28.535678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f628fd-b494-406c-a7c7-41c9fcb60983 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:28.537304 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 421/838] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:28.542825 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d2bec-41d7-4d36-b476-b3c793165a99 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:28.543251 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d2bec-41d7-4d36-b476-b3c793165a99 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:28.543449 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d2bec-41d7-4d36-b476-b3c793165a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:28.560294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1e3d2bec-41d7-4d36-b476-b3c793165a99 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:28.561482 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1e3d2bec-41d7-4d36-b476-b3c793165a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:28.602491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a264221-7f37-4db7-9aba-a23adef4ae63 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:28.602807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a264221-7f37-4db7-9aba-a23adef4ae63 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:28.603051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a264221-7f37-4db7-9aba-a23adef4ae63 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:28.604700 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 422/839] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:56:28.609939 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636ebbd-b153-4248-a6c7-71dc2521c7e6 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:28.610229 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636ebbd-b153-4248-a6c7-71dc2521c7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:28.610508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636ebbd-b153-4248-a6c7-71dc2521c7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:28.626795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9636ebbd-b153-4248-a6c7-71dc2521c7e6 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:28.628004 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9636ebbd-b153-4248-a6c7-71dc2521c7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:28.633743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 418/840] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:28.657437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d2bec-41d7-4d36-b476-b3c793165a99 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:28.701651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 423/841] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:28.723967 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636ebbd-b153-4248-a6c7-71dc2521c7e6 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:28.760098 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d2bec-41d7-4d36-b476-b3c793165a99 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:28.768898 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:28.796190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1131e2a-d5dd-4938-9e2f-f3adf22134f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:28.796929 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1131e2a-d5dd-4938-9e2f-f3adf22134f6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:28.797209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1131e2a-d5dd-4938-9e2f-f3adf22134f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:28.797487 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1131e2a-d5dd-4938-9e2f-f3adf22134f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:28.811444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1131e2a-d5dd-4938-9e2f-f3adf22134f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:28.812058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b1131e2a-d5dd-4938-9e2f-f3adf22134f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:28.820809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:28.822139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:28.822964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:28.822964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:28.822964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:28.822964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:28.822964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:28.822964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:28.822964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:28.822964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:28.822964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:28.822964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:28.822964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:28.822964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:28.825715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 419/842] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:56:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:28.828264 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1131e2a-d5dd-4938-9e2f-f3adf22134f6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:28.838430 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636ebbd-b153-4248-a6c7-71dc2521c7e6 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:28.876032 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ed078-72d4-44a2-ae3a-eef1770903cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:28.877248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ed078-72d4-44a2-ae3a-eef1770903cd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:28.877908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ed078-72d4-44a2-ae3a-eef1770903cd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:28.878404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ed078-72d4-44a2-ae3a-eef1770903cd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:28.879275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ed078-72d4-44a2-ae3a-eef1770903cd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:28.880999 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ed078-72d4-44a2-ae3a-eef1770903cd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-cd4ed078-72d4-44a2-ae3a-eef1770903cd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:28.894420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:28.895431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:56:28.896294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:28.901704 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 424/843] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:56:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:28.907125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ed078-72d4-44a2-ae3a-eef1770903cd None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:28.951360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1131e2a-d5dd-4938-9e2f-f3adf22134f6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:28.994239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f983691-915e-44f2-88d3-00ee777fc275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:28.995508 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f983691-915e-44f2-88d3-00ee777fc275 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:28.995977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f983691-915e-44f2-88d3-00ee777fc275 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:28.996429 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f983691-915e-44f2-88d3-00ee777fc275 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:28.997658 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f983691-915e-44f2-88d3-00ee777fc275 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:28.998190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f983691-915e-44f2-88d3-00ee777fc275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:28.999942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f983691-915e-44f2-88d3-00ee777fc275 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:29.013662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 420/844] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:56:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:29.017049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f983691-915e-44f2-88d3-00ee777fc275 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:29.023551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ed078-72d4-44a2-ae3a-eef1770903cd None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:29.054994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:29.059317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dad037-62eb-4efd-99d7-07863f8e2117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:29.059996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dad037-62eb-4efd-99d7-07863f8e2117 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:29.060279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dad037-62eb-4efd-99d7-07863f8e2117 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:29.060558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dad037-62eb-4efd-99d7-07863f8e2117 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:29.061348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dad037-62eb-4efd-99d7-07863f8e2117 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:29.061579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dad037-62eb-4efd-99d7-07863f8e2117 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:29.063949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dad037-62eb-4efd-99d7-07863f8e2117 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:29.074085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 425/845] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:56:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:29.075449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dad037-62eb-4efd-99d7-07863f8e2117 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:29.089914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:29.144038 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f983691-915e-44f2-88d3-00ee777fc275 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:29.182157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67529efc-c706-44c4-9718-c0e941cb45ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:29.182956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67529efc-c706-44c4-9718-c0e941cb45ec None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:29.183247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67529efc-c706-44c4-9718-c0e941cb45ec None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:29.183521 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67529efc-c706-44c4-9718-c0e941cb45ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:29.194106 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67529efc-c706-44c4-9718-c0e941cb45ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:29.195570 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67529efc-c706-44c4-9718-c0e941cb45ec None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:29.198138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dad037-62eb-4efd-99d7-07863f8e2117 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-67529efc-c706-44c4-9718-c0e941cb45ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:29.212650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:29.213551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:29.214511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:29.214511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:29.214511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:29.214511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:29.214511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 421/846] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:29.215492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fec144-7c41-4cee-bc74-76274df439e5 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:29.215778 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fec144-7c41-4cee-bc74-76274df439e5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:29.216052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fec144-7c41-4cee-bc74-76274df439e5 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:29.217433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 422/847] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:29.222097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fec144-7c41-4cee-bc74-76274df439e5 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:29.239815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755dfaf-a360-4bc3-99e4-0f0f35ffc69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:29.240488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755dfaf-a360-4bc3-99e4-0f0f35ffc69a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:29.240775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755dfaf-a360-4bc3-99e4-0f0f35ffc69a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:29.241053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755dfaf-a360-4bc3-99e4-0f0f35ffc69a None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:29.241800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755dfaf-a360-4bc3-99e4-0f0f35ffc69a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:29.242110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755dfaf-a360-4bc3-99e4-0f0f35ffc69a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:29.243497 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755dfaf-a360-4bc3-99e4-0f0f35ffc69a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:29.248052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 426/848] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:56:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:29.248052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d51f4d-fac1-43bf-bd13-72e0629b9e18 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:29.248052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d51f4d-fac1-43bf-bd13-72e0629b9e18 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:29.248052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d51f4d-fac1-43bf-bd13-72e0629b9e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:29.278401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a9d51f4d-fac1-43bf-bd13-72e0629b9e18 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:29.279669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a9d51f4d-fac1-43bf-bd13-72e0629b9e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:29.333439 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:29.364872 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fec144-7c41-4cee-bc74-76274df439e5 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:29.372940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 427/849] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:29.394173 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d51f4d-fac1-43bf-bd13-72e0629b9e18 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:29.396348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:29.403438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d7584-65c7-4480-8432-20819d68e9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:29.404118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d7584-65c7-4480-8432-20819d68e9bf None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:29.404399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d7584-65c7-4480-8432-20819d68e9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:29.404673 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d7584-65c7-4480-8432-20819d68e9bf None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:29.406109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d7584-65c7-4480-8432-20819d68e9bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:29.406407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d7584-65c7-4480-8432-20819d68e9bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:29.408241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d7584-65c7-4480-8432-20819d68e9bf None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:29.423691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 423/850] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:29.429842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d7584-65c7-4480-8432-20819d68e9bf None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:29.515215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d51f4d-fac1-43bf-bd13-72e0629b9e18 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:29.553390 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1085a-4cca-4c6f-a23f-d4b7234b0406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:29.554401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1085a-4cca-4c6f-a23f-d4b7234b0406 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:29.554770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1085a-4cca-4c6f-a23f-d4b7234b0406 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:29.555135 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1085a-4cca-4c6f-a23f-d4b7234b0406 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:29.560368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d7584-65c7-4480-8432-20819d68e9bf None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:29.567717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1085a-4cca-4c6f-a23f-d4b7234b0406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-8fb1085a-4cca-4c6f-a23f-d4b7234b0406 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:29.575727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:29.576874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:29.578428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:29.578428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:29.578428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:29.578428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:29.578428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:29.578428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:29.578428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:29.578428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:29.578428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:29.578428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:29.578428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:56:29.578428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:29.579830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 428/851] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:56:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:29.585200 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1085a-4cca-4c6f-a23f-d4b7234b0406 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:29.604479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250e381-b3d3-4ad0-8044-16021cd9b8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:29.605718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250e381-b3d3-4ad0-8044-16021cd9b8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:29.606206 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250e381-b3d3-4ad0-8044-16021cd9b8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:29.606679 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250e381-b3d3-4ad0-8044-16021cd9b8b2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:29.615592 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250e381-b3d3-4ad0-8044-16021cd9b8b2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:29.617564 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250e381-b3d3-4ad0-8044-16021cd9b8b2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:29.620042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-4250e381-b3d3-4ad0-8044-16021cd9b8b2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:29.635400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:29.636425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:29.637471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:29.637471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:29.637471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:29.637471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:29.637471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:29.637471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:29.637471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:29.637471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:29.637471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:56:29.637471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:29.640502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 424/852] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:56:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:29.650820 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250e381-b3d3-4ad0-8044-16021cd9b8b2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:29.768899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250e381-b3d3-4ad0-8044-16021cd9b8b2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:29.800159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:29.807048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69749698-3c32-46ec-852f-9c7d0687296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:29.808011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69749698-3c32-46ec-852f-9c7d0687296c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:29.808516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69749698-3c32-46ec-852f-9c7d0687296c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:29.809068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69749698-3c32-46ec-852f-9c7d0687296c None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:29.810267 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69749698-3c32-46ec-852f-9c7d0687296c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:29.810841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69749698-3c32-46ec-852f-9c7d0687296c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:29.812613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69749698-3c32-46ec-852f-9c7d0687296c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:29.833292 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 425/853] 10.222.0.79 () {56 vars in 1135 bytes} [Mon Jul 4 09:56:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:29.839269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69749698-3c32-46ec-852f-9c7d0687296c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:29.845786 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1085a-4cca-4c6f-a23f-d4b7234b0406 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:29.898338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773e520-6b51-47c2-b9bf-f38f4504ac09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:29.899225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773e520-6b51-47c2-b9bf-f38f4504ac09 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:29.899637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773e520-6b51-47c2-b9bf-f38f4504ac09 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:29.899978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773e520-6b51-47c2-b9bf-f38f4504ac09 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:29.900719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773e520-6b51-47c2-b9bf-f38f4504ac09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:29.901083 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773e520-6b51-47c2-b9bf-f38f4504ac09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:29.902849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773e520-6b51-47c2-b9bf-f38f4504ac09 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:29.912240 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 429/854] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:56:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:29.913488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a7ff9-724e-477a-8b82-13a74a74b56e None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:29.913807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a7ff9-724e-477a-8b82-13a74a74b56e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:29.914089 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a7ff9-724e-477a-8b82-13a74a74b56e None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:29.914915 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:29.915439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 430/855] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:29.918619 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a7ff9-724e-477a-8b82-13a74a74b56e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:29.961508 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69749698-3c32-46ec-852f-9c7d0687296c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:29.997063 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c8095-78f6-4cf8-8af8-6e05ac6f21e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:29.997747 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c8095-78f6-4cf8-8af8-6e05ac6f21e4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:29.998033 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c8095-78f6-4cf8-8af8-6e05ac6f21e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:29.998310 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c8095-78f6-4cf8-8af8-6e05ac6f21e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:30.027556 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a7ff9-724e-477a-8b82-13a74a74b56e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:30.053671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:30.066516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7557adf-dc6b-4119-903b-e4101a41a09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:30.067548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7557adf-dc6b-4119-903b-e4101a41a09d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:30.067834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7557adf-dc6b-4119-903b-e4101a41a09d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:30.068109 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7557adf-dc6b-4119-903b-e4101a41a09d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:30.072604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c8095-78f6-4cf8-8af8-6e05ac6f21e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:30.074225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c8095-78f6-4cf8-8af8-6e05ac6f21e4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:30.085189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7557adf-dc6b-4119-903b-e4101a41a09d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:30.087007 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7557adf-dc6b-4119-903b-e4101a41a09d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c7557adf-dc6b-4119-903b-e4101a41a09d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:30.105571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:30.106604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:30.107491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:30.107491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:30.107491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:30.107491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:30.107491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 431/856] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:30.108322 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa979ca-53ab-4523-a45b-1370842abb83 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:30.108322 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa979ca-53ab-4523-a45b-1370842abb83 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:30.108472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa979ca-53ab-4523-a45b-1370842abb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:30.125652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2aa979ca-53ab-4523-a45b-1370842abb83 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:30.126879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2aa979ca-53ab-4523-a45b-1370842abb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:30.146049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:30.197605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 426/857] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:56:30.198498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c8095-78f6-4cf8-8af8-6e05ac6f21e4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:30.221684 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 432/858] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:30.221684 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa979ca-53ab-4523-a45b-1370842abb83 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:30.314431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c8095-78f6-4cf8-8af8-6e05ac6f21e4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:30.334725 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa979ca-53ab-4523-a45b-1370842abb83 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:30.342559 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:30.351780 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82587e92-4548-4144-b4e1-96d807d8daf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:30.352463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82587e92-4548-4144-b4e1-96d807d8daf3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:30.352745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82587e92-4548-4144-b4e1-96d807d8daf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:30.353021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82587e92-4548-4144-b4e1-96d807d8daf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:30.353773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82587e92-4548-4144-b4e1-96d807d8daf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:30.354080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82587e92-4548-4144-b4e1-96d807d8daf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:30.355474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82587e92-4548-4144-b4e1-96d807d8daf3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:30.372115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 427/859] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:30.372893 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82587e92-4548-4144-b4e1-96d807d8daf3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:30.375434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599dec7-dcba-40a6-8274-6b709f823e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:30.376243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599dec7-dcba-40a6-8274-6b709f823e50 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:30.376524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599dec7-dcba-40a6-8274-6b709f823e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:30.376797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599dec7-dcba-40a6-8274-6b709f823e50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:30.387187 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599dec7-dcba-40a6-8274-6b709f823e50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:30.388632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599dec7-dcba-40a6-8274-6b709f823e50 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:30.392642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-e599dec7-dcba-40a6-8274-6b709f823e50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:30.408204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:30.409188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:30.410142 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:30.410142 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:30.410142 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:30.410142 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:30.410142 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 433/860] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:30.410435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599dec7-dcba-40a6-8274-6b709f823e50 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:30.496196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82587e92-4548-4144-b4e1-96d807d8daf3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:30.518288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599dec7-dcba-40a6-8274-6b709f823e50 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:30.536714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f46d8-d98e-49c9-8189-d5ffd6e47d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:30.546225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f46d8-d98e-49c9-8189-d5ffd6e47d0d None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:30.546770 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f46d8-d98e-49c9-8189-d5ffd6e47d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:30.546770 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f46d8-d98e-49c9-8189-d5ffd6e47d0d None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:30.547358 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f46d8-d98e-49c9-8189-d5ffd6e47d0d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:30.548753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f46d8-d98e-49c9-8189-d5ffd6e47d0d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:30.555261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c4e6c-784d-4a86-8217-48aeaa7994e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:30.556593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c4e6c-784d-4a86-8217-48aeaa7994e3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:30.557156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c4e6c-784d-4a86-8217-48aeaa7994e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:30.557578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c4e6c-784d-4a86-8217-48aeaa7994e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:30.567513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c4e6c-784d-4a86-8217-48aeaa7994e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:30.569213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c4e6c-784d-4a86-8217-48aeaa7994e3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:30.574928 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 428/861] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:56:30 2022] POST /identity/v3/services => generated 269 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:56:30.584633 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f46d8-d98e-49c9-8189-d5ffd6e47d0d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-a41c4e6c-784d-4a86-8217-48aeaa7994e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:30.587212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:30.588195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:30.589120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:30.589120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:30.589120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:30.589120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:30.589120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:30.589120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:30.589120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:30.589120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:30.589120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:56:30.589120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:30.591888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 434/862] 10.222.0.79 () {56 vars in 1117 bytes} [Mon Jul 4 09:56:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:30.598299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c4e6c-784d-4a86-8217-48aeaa7994e3 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:30.627495 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:30.694358 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:30.736777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c4e6c-784d-4a86-8217-48aeaa7994e3 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:30.738368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f46d8-d98e-49c9-8189-d5ffd6e47d0d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:30.780231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7a855-7a77-4cfd-9b20-c813e51b8131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:30.781000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8169044-b4ab-48ae-90b3-385fc4a7eb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:30.781275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7a855-7a77-4cfd-9b20-c813e51b8131 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:30.781566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7a855-7a77-4cfd-9b20-c813e51b8131 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:30.781887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7a855-7a77-4cfd-9b20-c813e51b8131 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:30.782572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8169044-b4ab-48ae-90b3-385fc4a7eb9e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:30.782744 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7a855-7a77-4cfd-9b20-c813e51b8131 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:30.783051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7a855-7a77-4cfd-9b20-c813e51b8131 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:30.783402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8169044-b4ab-48ae-90b3-385fc4a7eb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:30.783894 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8169044-b4ab-48ae-90b3-385fc4a7eb9e None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:30.784502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7a855-7a77-4cfd-9b20-c813e51b8131 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:30.785012 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8169044-b4ab-48ae-90b3-385fc4a7eb9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:30.785559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8169044-b4ab-48ae-90b3-385fc4a7eb9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:30.787164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8169044-b4ab-48ae-90b3-385fc4a7eb9e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:30.800014 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 429/863] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:30.805296 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 435/864] 10.222.0.79 () {56 vars in 1149 bytes} [Mon Jul 4 09:56:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:30.805428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8169044-b4ab-48ae-90b3-385fc4a7eb9e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:30.811006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7a855-7a77-4cfd-9b20-c813e51b8131 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:30.900386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:30.926171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7a855-7a77-4cfd-9b20-c813e51b8131 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:30.934887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8169044-b4ab-48ae-90b3-385fc4a7eb9e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:30.962924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bc590-3bfb-46f9-ae85-8b31fecef2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:30.963595 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bc590-3bfb-46f9-ae85-8b31fecef2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:30.964044 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bc590-3bfb-46f9-ae85-8b31fecef2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:30.964421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bc590-3bfb-46f9-ae85-8b31fecef2d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:30.970458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:30.978395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91517790-7cdb-458a-87d9-39ea9311a3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:30.979998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91517790-7cdb-458a-87d9-39ea9311a3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:30.980680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91517790-7cdb-458a-87d9-39ea9311a3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:30.981356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91517790-7cdb-458a-87d9-39ea9311a3f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:30.989824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91517790-7cdb-458a-87d9-39ea9311a3f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:30.991609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91517790-7cdb-458a-87d9-39ea9311a3f1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-91517790-7cdb-458a-87d9-39ea9311a3f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:31.007182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:31.008108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:31.008997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:31.008997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:31.008997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:31.008997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:31.008997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:31.008997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:31.008997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:31.008997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:31.011462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 430/865] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:31.017959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91517790-7cdb-458a-87d9-39ea9311a3f1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:31.052191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bc590-3bfb-46f9-ae85-8b31fecef2d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:31.054187 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bc590-3bfb-46f9-ae85-8b31fecef2d9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:31.142004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91517790-7cdb-458a-87d9-39ea9311a3f1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:31.162714 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:31.181088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ed574-5ae5-4e31-98cf-e9d8bd6900fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:31.182300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ed574-5ae5-4e31-98cf-e9d8bd6900fa None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:31.183052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ed574-5ae5-4e31-98cf-e9d8bd6900fa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:31.183772 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ed574-5ae5-4e31-98cf-e9d8bd6900fa None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:31.184740 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ed574-5ae5-4e31-98cf-e9d8bd6900fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:31.185370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ed574-5ae5-4e31-98cf-e9d8bd6900fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:31.188075 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ed574-5ae5-4e31-98cf-e9d8bd6900fa None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:31.197860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 436/866] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:30 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 09:56:31.198963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bc590-3bfb-46f9-ae85-8b31fecef2d9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:31.215502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 431/867] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:31.216506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ed574-5ae5-4e31-98cf-e9d8bd6900fa None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:31.271928 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:31.309210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bc590-3bfb-46f9-ae85-8b31fecef2d9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:31.322603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ed574-5ae5-4e31-98cf-e9d8bd6900fa None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:31.349499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7c396-6116-4884-b954-6cbd9c60c149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:31.350337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7c396-6116-4884-b954-6cbd9c60c149 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:31.350961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7c396-6116-4884-b954-6cbd9c60c149 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:31.351425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7c396-6116-4884-b954-6cbd9c60c149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:31.360396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01545b-8b56-484a-b6cf-3a4d04be6512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:31.361057 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01545b-8b56-484a-b6cf-3a4d04be6512 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:31.361340 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01545b-8b56-484a-b6cf-3a4d04be6512 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:31.361642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01545b-8b56-484a-b6cf-3a4d04be6512 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:31.362524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01545b-8b56-484a-b6cf-3a4d04be6512 None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=7ec37893ec8c480990c17cb03cb0ca19) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:31.362832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01545b-8b56-484a-b6cf-3a4d04be6512 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:31.363673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7c396-6116-4884-b954-6cbd9c60c149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:31.364288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01545b-8b56-484a-b6cf-3a4d04be6512 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:31.365115 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7c396-6116-4884-b954-6cbd9c60c149 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:31.374124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 432/868] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:31 2022] GET /identity/v3/role_assignments?user.id=7ec37893ec8c480990c17cb03cb0ca19&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=a3de9df253294d069dc1867fa26600c3 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:31.374124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcb93a-a0db-424e-b546-c42ae971e04d None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:31.374124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcb93a-a0db-424e-b546-c42ae971e04d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:31.374124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcb93a-a0db-424e-b546-c42ae971e04d None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:31.374711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 433/869] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-f5d7c396-6116-4884-b954-6cbd9c60c149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:31.386326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:31.387300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:31.388206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:31.388206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:31.388206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:31.388206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:31.388428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b9e69-1d5b-4a8e-8094-152f350a3b29 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:31.388428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b9e69-1d5b-4a8e-8094-152f350a3b29 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:31.388428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b9e69-1d5b-4a8e-8094-152f350a3b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:31.395595 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 437/870] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:31.398814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7c396-6116-4884-b954-6cbd9c60c149 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:31.410666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-136b9e69-1d5b-4a8e-8094-152f350a3b29 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:31.411901 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-136b9e69-1d5b-4a8e-8094-152f350a3b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:31.431645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:31.506238 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 434/871] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:31 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:31.517187 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7c396-6116-4884-b954-6cbd9c60c149 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:31.528726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b9e69-1d5b-4a8e-8094-152f350a3b29 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:31.560058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2ed02-18bb-440f-b7d2-3eef2f8205b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:31.560843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2ed02-18bb-440f-b7d2-3eef2f8205b0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:31.561194 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2ed02-18bb-440f-b7d2-3eef2f8205b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:31.561566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2ed02-18bb-440f-b7d2-3eef2f8205b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:31.563055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2ed02-18bb-440f-b7d2-3eef2f8205b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:31.563709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2ed02-18bb-440f-b7d2-3eef2f8205b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:31.565097 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2ed02-18bb-440f-b7d2-3eef2f8205b0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:31.577072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 438/872] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:31.584384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2ed02-18bb-440f-b7d2-3eef2f8205b0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:31.591872 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:31.660076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b9e69-1d5b-4a8e-8094-152f350a3b29 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:31.688554 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:31.708277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2ed02-18bb-440f-b7d2-3eef2f8205b0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:31.718154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07786e2-2d11-4399-95cd-7cf4b546a772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:31.718154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07786e2-2d11-4399-95cd-7cf4b546a772 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:31.718154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07786e2-2d11-4399-95cd-7cf4b546a772 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:31.718154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07786e2-2d11-4399-95cd-7cf4b546a772 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:31.718580 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07786e2-2d11-4399-95cd-7cf4b546a772 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:31.718580 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07786e2-2d11-4399-95cd-7cf4b546a772 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:31.739425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 435/873] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:56:31 2022] POST /identity/v3/services => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:56:31.758503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4aa80-6b49-432e-a336-24c35a17ea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:31.759349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4aa80-6b49-432e-a336-24c35a17ea47 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:31.759633 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4aa80-6b49-432e-a336-24c35a17ea47 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:31.759912 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4aa80-6b49-432e-a336-24c35a17ea47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:31.769447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4aa80-6b49-432e-a336-24c35a17ea47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:31.771340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4aa80-6b49-432e-a336-24c35a17ea47 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-73c4aa80-6b49-432e-a336-24c35a17ea47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:31.788432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:31.789410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:31.790416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:31.790416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:31.790416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:31.790416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:31.790416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:31.790416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:31.790416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:31.790416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:31.790416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 439/874] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:31.796409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07786e2-2d11-4399-95cd-7cf4b546a772 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:31.912169 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07786e2-2d11-4399-95cd-7cf4b546a772 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:31.919676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:31.948565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc172c18-03c5-404b-8103-2e3537b95f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:31.949164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc172c18-03c5-404b-8103-2e3537b95f36 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:31.949452 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc172c18-03c5-404b-8103-2e3537b95f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:31.949763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc172c18-03c5-404b-8103-2e3537b95f36 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:31.950561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc172c18-03c5-404b-8103-2e3537b95f36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:31.950871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc172c18-03c5-404b-8103-2e3537b95f36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:31.952390 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc172c18-03c5-404b-8103-2e3537b95f36 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:31.965212 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 436/875] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:31.971328 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4aa80-6b49-432e-a336-24c35a17ea47 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:32.077532 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4aa80-6b49-432e-a336-24c35a17ea47 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:32.112882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783523db-4b07-4d30-960f-59b782ccac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:32.113548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783523db-4b07-4d30-960f-59b782ccac48 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:32.113862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783523db-4b07-4d30-960f-59b782ccac48 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:32.114138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783523db-4b07-4d30-960f-59b782ccac48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:32.114974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783523db-4b07-4d30-960f-59b782ccac48 None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=5bdf239fa2bc4f1b9e484f916c438c1a) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:32.115288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783523db-4b07-4d30-960f-59b782ccac48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:32.116726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783523db-4b07-4d30-960f-59b782ccac48 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:32.123284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 440/876] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:31 2022] GET /identity/v3/role_assignments?user.id=5bdf239fa2bc4f1b9e484f916c438c1a&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:32.776042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7841-944c-4a16-80d7-b7302b26422e None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:32.776582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7841-944c-4a16-80d7-b7302b26422e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:32.776582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7841-944c-4a16-80d7-b7302b26422e None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:32.778220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 437/877] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:32.783621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02f6c9-8d6f-4102-b8ad-69aa78f7ad9c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:32.784010 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02f6c9-8d6f-4102-b8ad-69aa78f7ad9c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:32.784174 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02f6c9-8d6f-4102-b8ad-69aa78f7ad9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:32.801147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4b02f6c9-8d6f-4102-b8ad-69aa78f7ad9c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:32.802355 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4b02f6c9-8d6f-4102-b8ad-69aa78f7ad9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:32.804281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:32.872467 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 441/878] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:32.895642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d7841-944c-4a16-80d7-b7302b26422e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:33.002329 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d7841-944c-4a16-80d7-b7302b26422e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:33.036477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff84339-226a-43ba-811d-0f4bb060446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:33.037218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff84339-226a-43ba-811d-0f4bb060446e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:33.037502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff84339-226a-43ba-811d-0f4bb060446e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:33.037803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff84339-226a-43ba-811d-0f4bb060446e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:33.038448 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff84339-226a-43ba-811d-0f4bb060446e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:33.039845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff84339-226a-43ba-811d-0f4bb060446e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-cff84339-226a-43ba-811d-0f4bb060446e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:33.052010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:33.052938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:33.053852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 438/879] 10.222.0.79 () {56 vars in 1132 bytes} [Mon Jul 4 09:56:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:33.061273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02f6c9-8d6f-4102-b8ad-69aa78f7ad9c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:33.162624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02f6c9-8d6f-4102-b8ad-69aa78f7ad9c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:33.198047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d230a3-fc8b-4245-8ef7-92d932cb0339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:33.198717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d230a3-fc8b-4245-8ef7-92d932cb0339 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:33.198999 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d230a3-fc8b-4245-8ef7-92d932cb0339 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:33.199282 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d230a3-fc8b-4245-8ef7-92d932cb0339 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:33.199987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d230a3-fc8b-4245-8ef7-92d932cb0339 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:33.200297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d230a3-fc8b-4245-8ef7-92d932cb0339 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:33.201695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d230a3-fc8b-4245-8ef7-92d932cb0339 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:33.206918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 442/880] 10.222.0.79 () {56 vars in 1126 bytes} [Mon Jul 4 09:56:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:33.212630 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff84339-226a-43ba-811d-0f4bb060446e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:33.244852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:33.317840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff84339-226a-43ba-811d-0f4bb060446e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:33.359503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed5a5c-5d46-457a-8870-cf57c2e2fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:33.360182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed5a5c-5d46-457a-8870-cf57c2e2fb65 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:33.360463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed5a5c-5d46-457a-8870-cf57c2e2fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:33.360741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed5a5c-5d46-457a-8870-cf57c2e2fb65 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:33.361457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed5a5c-5d46-457a-8870-cf57c2e2fb65 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:33.361814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed5a5c-5d46-457a-8870-cf57c2e2fb65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:33.363207 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed5a5c-5d46-457a-8870-cf57c2e2fb65 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:33.368568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 439/881] 10.222.0.79 () {56 vars in 1126 bytes} [Mon Jul 4 09:56:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:33.374521 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d230a3-fc8b-4245-8ef7-92d932cb0339 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:33.377935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:33.480687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d230a3-fc8b-4245-8ef7-92d932cb0339 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:33.515388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1af95-0255-4e55-9248-2d97d49f2746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:33.516079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1af95-0255-4e55-9248-2d97d49f2746 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:33.516363 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1af95-0255-4e55-9248-2d97d49f2746 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:33.516638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1af95-0255-4e55-9248-2d97d49f2746 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:33.517429 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1af95-0255-4e55-9248-2d97d49f2746 None admin] RBAC: Adding query filter params (interface=public, service_id=19b64409a6fb438d893ced882cf9b417, region_id=RegionOne) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:33.517767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1af95-0255-4e55-9248-2d97d49f2746 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:33.519152 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1af95-0255-4e55-9248-2d97d49f2746 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:33.527355 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 443/882] 10.222.0.79 () {56 vars in 1251 bytes} [Mon Jul 4 09:56:33 2022] GET /identity/v3/endpoints?service_id=19b64409a6fb438d893ced882cf9b417&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:33.538731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed5a5c-5d46-457a-8870-cf57c2e2fb65 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:33.582805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bb0db-0c7c-48c2-93dc-a55f526b07b9 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:33.582980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bb0db-0c7c-48c2-93dc-a55f526b07b9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:33.583271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bb0db-0c7c-48c2-93dc-a55f526b07b9 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:33.584903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 444/883] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:56:33.589834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2e997-4990-4a88-8921-9aee96d43182 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:33.590128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2e997-4990-4a88-8921-9aee96d43182 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:33.590413 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2e997-4990-4a88-8921-9aee96d43182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:33.613708 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-92c2e997-4990-4a88-8921-9aee96d43182 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:33.614913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-92c2e997-4990-4a88-8921-9aee96d43182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:33.648008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:33.659511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed5a5c-5d46-457a-8870-cf57c2e2fb65 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:33.690837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:33.704949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087d520-0831-47e1-ba24-344c37629f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:33.705936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 445/884] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:33.706318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087d520-0831-47e1-ba24-344c37629f96 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:33.706794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087d520-0831-47e1-ba24-344c37629f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:33.707262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087d520-0831-47e1-ba24-344c37629f96 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:33.708133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087d520-0831-47e1-ba24-344c37629f96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:33.709744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087d520-0831-47e1-ba24-344c37629f96 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:33.716199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 440/885] 10.222.0.79 () {56 vars in 1087 bytes} [Mon Jul 4 09:56:33 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:56:33.724181 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2e997-4990-4a88-8921-9aee96d43182 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:33.841048 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2e997-4990-4a88-8921-9aee96d43182 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:33.880449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8fbb17-618d-4b27-99ae-e4afa7d17bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:33.881272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fbb17-618d-4b27-99ae-e4afa7d17bcd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:33.881555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fbb17-618d-4b27-99ae-e4afa7d17bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:33.881863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fbb17-618d-4b27-99ae-e4afa7d17bcd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:33.882598 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8fbb17-618d-4b27-99ae-e4afa7d17bcd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:33.884043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8fbb17-618d-4b27-99ae-e4afa7d17bcd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:33.889436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39d03f-81b5-434e-8802-d310ca7b89b4 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:33.890052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39d03f-81b5-434e-8802-d310ca7b89b4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:33.890751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39d03f-81b5-434e-8802-d310ca7b89b4 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:33.892164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 441/886] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-5c8fbb17-618d-4b27-99ae-e4afa7d17bcd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:33.896844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:33.897890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:33.898729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:33.899675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fe17c-59a5-44e1-9164-5a76eab42d65 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:33.899675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fe17c-59a5-44e1-9164-5a76eab42d65 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:33.899675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fe17c-59a5-44e1-9164-5a76eab42d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:33.900000 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 446/887] 10.222.0.79 () {56 vars in 1126 bytes} [Mon Jul 4 09:56:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:33.905309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8fbb17-618d-4b27-99ae-e4afa7d17bcd None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:33.919451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ac3fe17c-59a5-44e1-9164-5a76eab42d65 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:33.920653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ac3fe17c-59a5-44e1-9164-5a76eab42d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:33.971419 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:33.998653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 442/888] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:34.023655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8fbb17-618d-4b27-99ae-e4afa7d17bcd None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:34.025765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fe17c-59a5-44e1-9164-5a76eab42d65 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:34.065224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af195f-7167-4621-8a88-adef6cd78d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:34.066799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af195f-7167-4621-8a88-adef6cd78d16 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:34.067150 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af195f-7167-4621-8a88-adef6cd78d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:34.067432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af195f-7167-4621-8a88-adef6cd78d16 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:34.068180 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af195f-7167-4621-8a88-adef6cd78d16 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:34.068495 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af195f-7167-4621-8a88-adef6cd78d16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:34.069903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af195f-7167-4621-8a88-adef6cd78d16 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:34.077702 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 447/889] 10.222.0.79 () {56 vars in 1122 bytes} [Mon Jul 4 09:56:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:34.082513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af195f-7167-4621-8a88-adef6cd78d16 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:34.135762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:34.151378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fe17c-59a5-44e1-9164-5a76eab42d65 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:34.193644 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585d18c-95ab-41ac-95ed-21ede3dc5bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:34.194937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585d18c-95ab-41ac-95ed-21ede3dc5bad None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:34.195297 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585d18c-95ab-41ac-95ed-21ede3dc5bad None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:34.195665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585d18c-95ab-41ac-95ed-21ede3dc5bad None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:34.196658 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585d18c-95ab-41ac-95ed-21ede3dc5bad None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:34.198315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585d18c-95ab-41ac-95ed-21ede3dc5bad None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-4585d18c-95ab-41ac-95ed-21ede3dc5bad None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:34.214203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:34.215234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:56:34.216058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:34.217976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af195f-7167-4621-8a88-adef6cd78d16 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:34.220882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 443/890] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:56:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:34.221917 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585d18c-95ab-41ac-95ed-21ede3dc5bad None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:34.259804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54b2be-39d7-44d7-ab27-0c5176a6cd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:34.260487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54b2be-39d7-44d7-ab27-0c5176a6cd43 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:34.260773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54b2be-39d7-44d7-ab27-0c5176a6cd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:34.261050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54b2be-39d7-44d7-ab27-0c5176a6cd43 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:34.261887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54b2be-39d7-44d7-ab27-0c5176a6cd43 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:34.262128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54b2be-39d7-44d7-ab27-0c5176a6cd43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:34.263513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54b2be-39d7-44d7-ab27-0c5176a6cd43 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:34.266024 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:34.278091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 448/891] 10.222.0.79 () {56 vars in 1122 bytes} [Mon Jul 4 09:56:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:34.348807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585d18c-95ab-41ac-95ed-21ede3dc5bad None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:34.387685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a8e6b-f90f-4a7e-b732-7580aa9e4ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:34.388752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a8e6b-f90f-4a7e-b732-7580aa9e4ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:34.389231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a8e6b-f90f-4a7e-b732-7580aa9e4ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:34.389812 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a8e6b-f90f-4a7e-b732-7580aa9e4ee2 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:34.390760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a8e6b-f90f-4a7e-b732-7580aa9e4ee2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:34.391319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a8e6b-f90f-4a7e-b732-7580aa9e4ee2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:34.392950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a8e6b-f90f-4a7e-b732-7580aa9e4ee2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:34.395291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f80f5-2f75-4e45-afab-694790346134 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:34.395590 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f80f5-2f75-4e45-afab-694790346134 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:34.396080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f80f5-2f75-4e45-afab-694790346134 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:34.397243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 449/892] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:34.400048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 444/893] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:56:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:34.402949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb19e92-e7fa-40b8-8dac-737592614f9a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:34.403246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb19e92-e7fa-40b8-8dac-737592614f9a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:34.403524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb19e92-e7fa-40b8-8dac-737592614f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:34.409401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a8e6b-f90f-4a7e-b732-7580aa9e4ee2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:34.425520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4cb19e92-e7fa-40b8-8dac-737592614f9a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:34.427365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4cb19e92-e7fa-40b8-8dac-737592614f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:34.431657 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:34.528712 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 450/894] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:34.547318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a8e6b-f90f-4a7e-b732-7580aa9e4ee2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:34.552100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb19e92-e7fa-40b8-8dac-737592614f9a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:34.589750 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7ff41-03d5-4ea8-9b87-756e8fe6fc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:34.591286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7ff41-03d5-4ea8-9b87-756e8fe6fc15 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:34.591573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7ff41-03d5-4ea8-9b87-756e8fe6fc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:34.591855 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7ff41-03d5-4ea8-9b87-756e8fe6fc15 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:34.592567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7ff41-03d5-4ea8-9b87-756e8fe6fc15 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:34.592924 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7ff41-03d5-4ea8-9b87-756e8fe6fc15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:34.594426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7ff41-03d5-4ea8-9b87-756e8fe6fc15 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:34.601644 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 445/895] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:56:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:34.607332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7ff41-03d5-4ea8-9b87-756e8fe6fc15 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:34.665037 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb19e92-e7fa-40b8-8dac-737592614f9a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:34.682883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:34.708265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ea4cd-6dbe-4a72-ab25-f701c707784d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:34.709361 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ea4cd-6dbe-4a72-ab25-f701c707784d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:34.709811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ea4cd-6dbe-4a72-ab25-f701c707784d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:34.710247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ea4cd-6dbe-4a72-ab25-f701c707784d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:34.720471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:34.722186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ea4cd-6dbe-4a72-ab25-f701c707784d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-aa5ea4cd-6dbe-4a72-ab25-f701c707784d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:34.731087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:34.732340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:34.733182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:34.733182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:34.733182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:34.733182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:34.733182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:34.733182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:34.733182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:34.733182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:34.733182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:34.733182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:34.733182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:56:34.733182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:34.740799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 451/896] 10.222.0.79 () {56 vars in 1120 bytes} [Mon Jul 4 09:56:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:34.741919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ea4cd-6dbe-4a72-ab25-f701c707784d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:34.744849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7ff41-03d5-4ea8-9b87-756e8fe6fc15 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:34.784017 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c6f1d-4bd9-41c7-b515-7d12bb0ea800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:34.784802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c6f1d-4bd9-41c7-b515-7d12bb0ea800 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:34.785157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c6f1d-4bd9-41c7-b515-7d12bb0ea800 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:34.785510 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c6f1d-4bd9-41c7-b515-7d12bb0ea800 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:34.786414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c6f1d-4bd9-41c7-b515-7d12bb0ea800 None admin] RBAC: Adding query filter params (interface=public, service_id=e12034e55b9b41bf943c24a742d58683, region_id=RegionOne) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:34.786814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c6f1d-4bd9-41c7-b515-7d12bb0ea800 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:34.788300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c6f1d-4bd9-41c7-b515-7d12bb0ea800 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:34.799095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 446/897] 10.222.0.79 () {56 vars in 1251 bytes} [Mon Jul 4 09:56:34 2022] GET /identity/v3/endpoints?service_id=e12034e55b9b41bf943c24a742d58683&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:34.800209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c6f1d-4bd9-41c7-b515-7d12bb0ea800 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:34.861063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ea4cd-6dbe-4a72-ab25-f701c707784d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:34.912365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c6d43-8af4-447f-9d63-3341f1199617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:34.913389 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c6d43-8af4-447f-9d63-3341f1199617 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:34.913916 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c6d43-8af4-447f-9d63-3341f1199617 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:34.914368 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c6d43-8af4-447f-9d63-3341f1199617 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:34.915344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c6d43-8af4-447f-9d63-3341f1199617 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:34.915824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c6d43-8af4-447f-9d63-3341f1199617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:34.916502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c6f1d-4bd9-41c7-b515-7d12bb0ea800 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:34.917981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c6d43-8af4-447f-9d63-3341f1199617 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:34.932247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 452/898] 10.222.0.79 () {56 vars in 1115 bytes} [Mon Jul 4 09:56:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:34.933479 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c6d43-8af4-447f-9d63-3341f1199617 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:34.957535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687f474-1e8e-452a-8621-64a0f71c0a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:34.958245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687f474-1e8e-452a-8621-64a0f71c0a03 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:34.958535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687f474-1e8e-452a-8621-64a0f71c0a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:34.958818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687f474-1e8e-452a-8621-64a0f71c0a03 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:34.959889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687f474-1e8e-452a-8621-64a0f71c0a03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:34.961480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687f474-1e8e-452a-8621-64a0f71c0a03 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:34.973706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 447/899] 10.222.0.79 () {56 vars in 1087 bytes} [Mon Jul 4 09:56:34 2022] GET /identity/v3/services => generated 1268 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:56:34.986216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:35.050729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c6d43-8af4-447f-9d63-3341f1199617 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:35.087532 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078a58f-9853-46d4-8c57-60adbbff628f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:35.088765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078a58f-9853-46d4-8c57-60adbbff628f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:35.089296 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078a58f-9853-46d4-8c57-60adbbff628f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:35.089816 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078a58f-9853-46d4-8c57-60adbbff628f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:35.102302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078a58f-9853-46d4-8c57-60adbbff628f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:35.104107 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078a58f-9853-46d4-8c57-60adbbff628f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-a078a58f-9853-46d4-8c57-60adbbff628f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:35.123516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:35.125052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:35.126070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:35.126070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:35.126070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:35.126070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:35.128502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 453/900] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:35.134675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687f474-1e8e-452a-8621-64a0f71c0a03 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:35.148533 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:35.246358 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687f474-1e8e-452a-8621-64a0f71c0a03 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:35.280394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa90c2-1ff6-43fc-9ce2-7337ecab78c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:35.281210 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa90c2-1ff6-43fc-9ce2-7337ecab78c4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:35.281495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa90c2-1ff6-43fc-9ce2-7337ecab78c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:35.281804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa90c2-1ff6-43fc-9ce2-7337ecab78c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:35.282529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa90c2-1ff6-43fc-9ce2-7337ecab78c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:35.282836 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa90c2-1ff6-43fc-9ce2-7337ecab78c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:35.284227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa90c2-1ff6-43fc-9ce2-7337ecab78c4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:35.297064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 448/901] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:35.302393 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078a58f-9853-46d4-8c57-60adbbff628f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:35.398238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:35.407835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078a58f-9853-46d4-8c57-60adbbff628f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:35.444000 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105002b1-a075-4ffd-b94f-5a682374564c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:35.444843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105002b1-a075-4ffd-b94f-5a682374564c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:35.445127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105002b1-a075-4ffd-b94f-5a682374564c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:35.445407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105002b1-a075-4ffd-b94f-5a682374564c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:35.454149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105002b1-a075-4ffd-b94f-5a682374564c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:35.455613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105002b1-a075-4ffd-b94f-5a682374564c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-105002b1-a075-4ffd-b94f-5a682374564c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:35.470989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:35.471952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:35.472866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:35.472866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:35.472866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:35.472866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:35.472866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:35.472866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:35.472866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:35.472866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:35.472866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:56:35.472866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:35.472866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 454/902] 10.222.0.79 () {56 vars in 1117 bytes} [Mon Jul 4 09:56:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:35.478620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa90c2-1ff6-43fc-9ce2-7337ecab78c4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:35.554010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:35.589051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa90c2-1ff6-43fc-9ce2-7337ecab78c4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:35.632880 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8503bb-705d-46c3-bd0b-ec82c3e0ab13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:35.633604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8503bb-705d-46c3-bd0b-ec82c3e0ab13 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:35.634077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8503bb-705d-46c3-bd0b-ec82c3e0ab13 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:35.634357 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8503bb-705d-46c3-bd0b-ec82c3e0ab13 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:35.635195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8503bb-705d-46c3-bd0b-ec82c3e0ab13 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:35.635502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8503bb-705d-46c3-bd0b-ec82c3e0ab13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:35.636953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8503bb-705d-46c3-bd0b-ec82c3e0ab13 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:35.655028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 449/903] 10.222.0.79 () {56 vars in 1149 bytes} [Mon Jul 4 09:56:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:35.661044 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105002b1-a075-4ffd-b94f-5a682374564c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:35.765855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105002b1-a075-4ffd-b94f-5a682374564c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:35.800549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f4cd8-915b-49c8-9644-4e7059108812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:35.801606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f4cd8-915b-49c8-9644-4e7059108812 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:35.802128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f4cd8-915b-49c8-9644-4e7059108812 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:35.802589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f4cd8-915b-49c8-9644-4e7059108812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:35.825336 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:35.833317 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695a61f-6382-4a15-8026-7d8a54543c19 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:35.833610 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695a61f-6382-4a15-8026-7d8a54543c19 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:35.834057 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695a61f-6382-4a15-8026-7d8a54543c19 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:35.838060 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 450/904] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:35.840940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd5faf-4b25-439b-b56a-128669091848 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:35.841234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd5faf-4b25-439b-b56a-128669091848 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:35.841518 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd5faf-4b25-439b-b56a-128669091848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:35.861364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-12fd5faf-4b25-439b-b56a-128669091848 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:35.862707 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-12fd5faf-4b25-439b-b56a-128669091848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:35.872421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f4cd8-915b-49c8-9644-4e7059108812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:35.873912 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f4cd8-915b-49c8-9644-4e7059108812 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:35.949772 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 451/905] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:35.973609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd5faf-4b25-439b-b56a-128669091848 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:35.983434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 455/906] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:35 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:56:35.989857 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f4cd8-915b-49c8-9644-4e7059108812 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:36.035908 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:36.069483 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:36.085803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd5faf-4b25-439b-b56a-128669091848 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:36.135614 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f4cd8-915b-49c8-9644-4e7059108812 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:36.138088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc25cc-f3b7-420b-a86a-57d4297faa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:36.139142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc25cc-f3b7-420b-a86a-57d4297faa2c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:36.139677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc25cc-f3b7-420b-a86a-57d4297faa2c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:36.140143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc25cc-f3b7-420b-a86a-57d4297faa2c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:36.141019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc25cc-f3b7-420b-a86a-57d4297faa2c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:36.142631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc25cc-f3b7-420b-a86a-57d4297faa2c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b3fc25cc-f3b7-420b-a86a-57d4297faa2c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:36.156308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:36.157282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:56:36.158152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:36.161120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 452/907] 10.222.0.79 () {56 vars in 1132 bytes} [Mon Jul 4 09:56:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:36.167125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc25cc-f3b7-420b-a86a-57d4297faa2c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:36.181665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15713397-8fc9-4bbb-8bc0-446f3c466fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:36.182747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15713397-8fc9-4bbb-8bc0-446f3c466fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:36.183040 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15713397-8fc9-4bbb-8bc0-446f3c466fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:36.183323 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15713397-8fc9-4bbb-8bc0-446f3c466fd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:36.198594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15713397-8fc9-4bbb-8bc0-446f3c466fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:36.200339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15713397-8fc9-4bbb-8bc0-446f3c466fd5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-15713397-8fc9-4bbb-8bc0-446f3c466fd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:36.228606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:36.229565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:36.230471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:36.230471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:36.230471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:36.230471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:36.231120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 456/908] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:36.236305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15713397-8fc9-4bbb-8bc0-446f3c466fd5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:36.287277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc25cc-f3b7-420b-a86a-57d4297faa2c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:36.308946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:36.325260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e659e-013b-4b97-8851-1a4cfd359c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:36.326302 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e659e-013b-4b97-8851-1a4cfd359c22 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:36.326790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e659e-013b-4b97-8851-1a4cfd359c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:36.327389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e659e-013b-4b97-8851-1a4cfd359c22 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:36.328354 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e659e-013b-4b97-8851-1a4cfd359c22 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:36.328951 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e659e-013b-4b97-8851-1a4cfd359c22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:36.330575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e659e-013b-4b97-8851-1a4cfd359c22 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:36.336324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 453/909] 10.222.0.79 () {56 vars in 1126 bytes} [Mon Jul 4 09:56:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:36.341731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e659e-013b-4b97-8851-1a4cfd359c22 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:36.372470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15713397-8fc9-4bbb-8bc0-446f3c466fd5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:36.375915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:36.409605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8552e21-8638-4b91-9c56-48e5c5f5899b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:36.410315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8552e21-8638-4b91-9c56-48e5c5f5899b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:36.410605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8552e21-8638-4b91-9c56-48e5c5f5899b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:36.410879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8552e21-8638-4b91-9c56-48e5c5f5899b None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:36.411635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8552e21-8638-4b91-9c56-48e5c5f5899b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:36.411922 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8552e21-8638-4b91-9c56-48e5c5f5899b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:36.413302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8552e21-8638-4b91-9c56-48e5c5f5899b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:36.424886 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 457/910] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:36.426414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a96f3-9955-40bd-b33d-4953f4b4bf57 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:36.426905 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a96f3-9955-40bd-b33d-4953f4b4bf57 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:36.427398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a96f3-9955-40bd-b33d-4953f4b4bf57 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:36.428856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 458/911] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:36.430172 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a96f3-9955-40bd-b33d-4953f4b4bf57 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:36.468775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e659e-013b-4b97-8851-1a4cfd359c22 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:36.501902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546dda8-eedc-45b7-986b-ca12431d8618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:36.502565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546dda8-eedc-45b7-986b-ca12431d8618 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:36.502843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546dda8-eedc-45b7-986b-ca12431d8618 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:36.503115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546dda8-eedc-45b7-986b-ca12431d8618 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:36.503827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546dda8-eedc-45b7-986b-ca12431d8618 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:36.504109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546dda8-eedc-45b7-986b-ca12431d8618 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:36.505488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546dda8-eedc-45b7-986b-ca12431d8618 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:36.510671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 454/912] 10.222.0.79 () {56 vars in 1126 bytes} [Mon Jul 4 09:56:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:36.511811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e8d4a-f1fe-4b6e-9e64-ee6b421a39ce None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:36.512093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e8d4a-f1fe-4b6e-9e64-ee6b421a39ce None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:36.512366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e8d4a-f1fe-4b6e-9e64-ee6b421a39ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:36.528352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-cd4e8d4a-f1fe-4b6e-9e64-ee6b421a39ce None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:36.529517 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-cd4e8d4a-f1fe-4b6e-9e64-ee6b421a39ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:36.544043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a96f3-9955-40bd-b33d-4953f4b4bf57 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:36.573199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:36.580883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40072679-3911-4cea-8ecd-d6d4d8b8a9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:36.581933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40072679-3911-4cea-8ecd-d6d4d8b8a9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:36.582381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40072679-3911-4cea-8ecd-d6d4d8b8a9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:36.582797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40072679-3911-4cea-8ecd-d6d4d8b8a9f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:36.590458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40072679-3911-4cea-8ecd-d6d4d8b8a9f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:36.592307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40072679-3911-4cea-8ecd-d6d4d8b8a9f8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:36.599276 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 455/913] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:36.600247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e8d4a-f1fe-4b6e-9e64-ee6b421a39ce None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-40072679-3911-4cea-8ecd-d6d4d8b8a9f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:36.606989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:36.607865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:36.608802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:36.608802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:36.608802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:36.608802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:36.608802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:36.608802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:36.608802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:36.608802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:36.609908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 459/914] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:36.613813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40072679-3911-4cea-8ecd-d6d4d8b8a9f8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:36.641891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:36.712818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e8d4a-f1fe-4b6e-9e64-ee6b421a39ce None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:36.722127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40072679-3911-4cea-8ecd-d6d4d8b8a9f8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:36.750704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef0776-d784-40a7-9285-6a965bdf4e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:36.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef0776-d784-40a7-9285-6a965bdf4e9d None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:36.751656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef0776-d784-40a7-9285-6a965bdf4e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:36.751932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef0776-d784-40a7-9285-6a965bdf4e9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:36.752668 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef0776-d784-40a7-9285-6a965bdf4e9d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:36.754122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef0776-d784-40a7-9285-6a965bdf4e9d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:36.757450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f87af2-d016-444a-b40f-04805ff21822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:36.758174 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f87af2-d016-444a-b40f-04805ff21822 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:36.758427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f87af2-d016-444a-b40f-04805ff21822 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:36.758705 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f87af2-d016-444a-b40f-04805ff21822 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:36.759528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f87af2-d016-444a-b40f-04805ff21822 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:36.759805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f87af2-d016-444a-b40f-04805ff21822 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:36.761254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f87af2-d016-444a-b40f-04805ff21822 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:36.778796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 460/915] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:36.778796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f87af2-d016-444a-b40f-04805ff21822 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:36.789650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 456/916] 10.222.0.79 () {60 vars in 1144 bytes} [Mon Jul 4 09:56:36 2022] POST /identity/v3/endpoints => generated 347 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:56:36.789650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef0776-d784-40a7-9285-6a965bdf4e9d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:36.822813 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:36.891030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:36.909995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f87af2-d016-444a-b40f-04805ff21822 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:36.937676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef0776-d784-40a7-9285-6a965bdf4e9d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:36.948610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48e907-d000-4e14-87d2-16ea04ef5d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:36.949415 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48e907-d000-4e14-87d2-16ea04ef5d56 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:36.949840 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48e907-d000-4e14-87d2-16ea04ef5d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:36.950214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48e907-d000-4e14-87d2-16ea04ef5d56 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:36.951038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48e907-d000-4e14-87d2-16ea04ef5d56 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:36.952502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48e907-d000-4e14-87d2-16ea04ef5d56 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:36.973325 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 461/917] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:56:36 2022] POST /identity/v3/services => generated 241 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:56:36.985968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c11e8f-071c-486a-a15e-3d6c120cf942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:36.986674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c11e8f-071c-486a-a15e-3d6c120cf942 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:36.987090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c11e8f-071c-486a-a15e-3d6c120cf942 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:36.987465 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c11e8f-071c-486a-a15e-3d6c120cf942 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:36.988411 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c11e8f-071c-486a-a15e-3d6c120cf942 None admin] RBAC: Adding query filter params (role.id=44488839b54a4c9197dca8c9e2226e84, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=5bdf239fa2bc4f1b9e484f916c438c1a) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:36.988801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c11e8f-071c-486a-a15e-3d6c120cf942 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:36.990428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c11e8f-071c-486a-a15e-3d6c120cf942 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:36.998407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 457/918] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:36 2022] GET /identity/v3/role_assignments?user.id=5bdf239fa2bc4f1b9e484f916c438c1a&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=44488839b54a4c9197dca8c9e2226e84 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:37.251443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb9aa-6886-4062-9eba-6e75078bc908 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:37.251626 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb9aa-6886-4062-9eba-6e75078bc908 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:37.251916 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb9aa-6886-4062-9eba-6e75078bc908 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:37.253634 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 462/919] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:37.259102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad643826-5c16-43d8-b39b-fb5702f0de90 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:37.259395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad643826-5c16-43d8-b39b-fb5702f0de90 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:37.259680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad643826-5c16-43d8-b39b-fb5702f0de90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:37.277928 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ad643826-5c16-43d8-b39b-fb5702f0de90 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:37.279135 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ad643826-5c16-43d8-b39b-fb5702f0de90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:37.356315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 458/920] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:37.384209 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb9aa-6886-4062-9eba-6e75078bc908 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:37.493811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb9aa-6886-4062-9eba-6e75078bc908 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:37.528823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94367a47-448d-485c-a7e5-9172bb8a07b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:37.530044 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94367a47-448d-485c-a7e5-9172bb8a07b3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:37.530558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94367a47-448d-485c-a7e5-9172bb8a07b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:37.531031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94367a47-448d-485c-a7e5-9172bb8a07b3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:37.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94367a47-448d-485c-a7e5-9172bb8a07b3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:37.533456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94367a47-448d-485c-a7e5-9172bb8a07b3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-94367a47-448d-485c-a7e5-9172bb8a07b3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:37.547959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:37.548973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:56:37.549836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:37.552700 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 463/921] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:56:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:37.558533 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad643826-5c16-43d8-b39b-fb5702f0de90 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:37.580960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:37.666205 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad643826-5c16-43d8-b39b-fb5702f0de90 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:37.702194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf1b3e-be96-4729-b186-6fdb19f55a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:37.703279 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf1b3e-be96-4729-b186-6fdb19f55a8c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:37.703790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf1b3e-be96-4729-b186-6fdb19f55a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:37.704263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf1b3e-be96-4729-b186-6fdb19f55a8c None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:37.705185 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf1b3e-be96-4729-b186-6fdb19f55a8c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:37.705708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf1b3e-be96-4729-b186-6fdb19f55a8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:37.707296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf1b3e-be96-4729-b186-6fdb19f55a8c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:37.713065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 459/922] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:56:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:37.718801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94367a47-448d-485c-a7e5-9172bb8a07b3 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:37.734117 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:37.828181 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94367a47-448d-485c-a7e5-9172bb8a07b3 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:37.862638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadbd07-4357-41aa-a9d1-93a71f3a0f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:37.863637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadbd07-4357-41aa-a9d1-93a71f3a0f87 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:37.864257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadbd07-4357-41aa-a9d1-93a71f3a0f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:37.864728 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadbd07-4357-41aa-a9d1-93a71f3a0f87 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:37.865659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadbd07-4357-41aa-a9d1-93a71f3a0f87 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:37.866197 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadbd07-4357-41aa-a9d1-93a71f3a0f87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:37.867807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadbd07-4357-41aa-a9d1-93a71f3a0f87 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:37.873187 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 464/923] 10.222.0.79 () {56 vars in 1114 bytes} [Mon Jul 4 09:56:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:37.879213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf1b3e-be96-4729-b186-6fdb19f55a8c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:37.968083 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:37.984986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf1b3e-be96-4729-b186-6fdb19f55a8c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:38.021348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f94773-8cf2-4dac-8186-922286b9e0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:38.022071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f94773-8cf2-4dac-8186-922286b9e0ad None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:38.022335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f94773-8cf2-4dac-8186-922286b9e0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:38.022605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f94773-8cf2-4dac-8186-922286b9e0ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:38.023387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f94773-8cf2-4dac-8186-922286b9e0ad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:38.024806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f94773-8cf2-4dac-8186-922286b9e0ad None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:38.050203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 460/924] 10.222.0.79 () {60 vars in 1144 bytes} [Mon Jul 4 09:56:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:56:39.000087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a85cf-b127-41a7-ba42-5609742f2b1b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.000623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a85cf-b127-41a7-ba42-5609742f2b1b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.000623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a85cf-b127-41a7-ba42-5609742f2b1b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.002358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 465/925] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:39.007833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a0b7e-15ed-4c6f-aa6b-b53260ddb27e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.008128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a0b7e-15ed-4c6f-aa6b-b53260ddb27e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.008411 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a0b7e-15ed-4c6f-aa6b-b53260ddb27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.025817 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7f3a0b7e-15ed-4c6f-aa6b-b53260ddb27e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:39.027104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7f3a0b7e-15ed-4c6f-aa6b-b53260ddb27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:39.070183 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296dce18-9d70-43ba-9831-08bbf074b6c4 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.070502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296dce18-9d70-43ba-9831-08bbf074b6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.070755 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296dce18-9d70-43ba-9831-08bbf074b6c4 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.072364 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 466/926] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:39.077871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc4bce-aac6-4956-97a6-5db3c220f5ab None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.078161 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc4bce-aac6-4956-97a6-5db3c220f5ab None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.078442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc4bce-aac6-4956-97a6-5db3c220f5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.095191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a4cc4bce-aac6-4956-97a6-5db3c220f5ab None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:39.096381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a4cc4bce-aac6-4956-97a6-5db3c220f5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:39.113279 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 461/927] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:39.136550 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a0b7e-15ed-4c6f-aa6b-b53260ddb27e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:39.185414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 467/928] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:39.187561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77a91e-7730-4e99-b879-9158ae9f27af None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.188087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77a91e-7730-4e99-b879-9158ae9f27af None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.188567 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77a91e-7730-4e99-b879-9158ae9f27af None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.190901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 468/929] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:39.199394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ca5a3-fcd1-4aab-9d11-805f79480fe9 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.199944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ca5a3-fcd1-4aab-9d11-805f79480fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.200424 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ca5a3-fcd1-4aab-9d11-805f79480fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.209904 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:39.216521 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:39.225020 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b67ca5a3-fcd1-4aab-9d11-805f79480fe9 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:39.226656 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b67ca5a3-fcd1-4aab-9d11-805f79480fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:39.258907 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a0b7e-15ed-4c6f-aa6b-b53260ddb27e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:39.295239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616f137-73d2-46a3-b187-e7e284a8507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:39.301763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616f137-73d2-46a3-b187-e7e284a8507f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.301763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616f137-73d2-46a3-b187-e7e284a8507f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.301763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616f137-73d2-46a3-b187-e7e284a8507f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.302069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 469/930] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:39.302147 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616f137-73d2-46a3-b187-e7e284a8507f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:39.302217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ca5a3-fcd1-4aab-9d11-805f79480fe9 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:39.302298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616f137-73d2-46a3-b187-e7e284a8507f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-2616f137-73d2-46a3-b187-e7e284a8507f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:39.320698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:39.321786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:56:39.322632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:39.323364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 462/931] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:39.323959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616f137-73d2-46a3-b187-e7e284a8507f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:39.406206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ca5a3-fcd1-4aab-9d11-805f79480fe9 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:39.436686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616f137-73d2-46a3-b187-e7e284a8507f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:39.437545 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:39.443152 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0868a61-879b-4b48-bc9c-e08f8dd171de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:39.444156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0868a61-879b-4b48-bc9c-e08f8dd171de None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.444775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0868a61-879b-4b48-bc9c-e08f8dd171de None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.445154 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0868a61-879b-4b48-bc9c-e08f8dd171de None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.445845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0868a61-879b-4b48-bc9c-e08f8dd171de None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:39.447343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0868a61-879b-4b48-bc9c-e08f8dd171de None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d0868a61-879b-4b48-bc9c-e08f8dd171de None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:39.460241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:39.461224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:39.462436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 470/932] 10.222.0.79 () {56 vars in 1126 bytes} [Mon Jul 4 09:56:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:39.463535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0868a61-879b-4b48-bc9c-e08f8dd171de None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:39.473058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:39.480041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7a07b-c080-444f-86cd-59066e006d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:39.480859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7a07b-c080-444f-86cd-59066e006d2b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.481146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7a07b-c080-444f-86cd-59066e006d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.481424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7a07b-c080-444f-86cd-59066e006d2b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.491650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7a07b-c080-444f-86cd-59066e006d2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-03f7a07b-c080-444f-86cd-59066e006d2b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:39.499841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:39.500763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:39.501597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:39.501597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:39.501597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:39.501597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:39.501597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:39.501597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:39.501597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:39.501597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:39.501597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:39.501597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:39.501597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:56:39.501597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:39.505959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 463/933] 10.222.0.79 () {56 vars in 1120 bytes} [Mon Jul 4 09:56:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:39.506823 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7a07b-c080-444f-86cd-59066e006d2b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:39.570622 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0868a61-879b-4b48-bc9c-e08f8dd171de None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:39.604978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbe090-0a28-4fb6-929c-e05dee730254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:39.605676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbe090-0a28-4fb6-929c-e05dee730254 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.605967 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbe090-0a28-4fb6-929c-e05dee730254 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.606241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbe090-0a28-4fb6-929c-e05dee730254 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.606948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbe090-0a28-4fb6-929c-e05dee730254 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:39.607254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbe090-0a28-4fb6-929c-e05dee730254 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:39.608913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbe090-0a28-4fb6-929c-e05dee730254 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:39.620314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 471/934] 10.222.0.79 () {56 vars in 1112 bytes} [Mon Jul 4 09:56:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:39.620314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbe090-0a28-4fb6-929c-e05dee730254 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:39.623131 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7a07b-c080-444f-86cd-59066e006d2b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:39.662472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db7d3d-3ed0-451d-8bba-f7e36fca4323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:39.663144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db7d3d-3ed0-451d-8bba-f7e36fca4323 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.663430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db7d3d-3ed0-451d-8bba-f7e36fca4323 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.663705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db7d3d-3ed0-451d-8bba-f7e36fca4323 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.664432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db7d3d-3ed0-451d-8bba-f7e36fca4323 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:39.664715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db7d3d-3ed0-451d-8bba-f7e36fca4323 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:39.666246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db7d3d-3ed0-451d-8bba-f7e36fca4323 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:39.676137 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 464/935] 10.222.0.79 () {56 vars in 1122 bytes} [Mon Jul 4 09:56:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:39.676137 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db7d3d-3ed0-451d-8bba-f7e36fca4323 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:39.690950 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:39.733281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:39.735689 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbe090-0a28-4fb6-929c-e05dee730254 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:39.769859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef9f2d-8d53-4b0b-b350-9a486aaef277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:39.770532 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef9f2d-8d53-4b0b-b350-9a486aaef277 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.770817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef9f2d-8d53-4b0b-b350-9a486aaef277 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.771093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef9f2d-8d53-4b0b-b350-9a486aaef277 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.771840 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef9f2d-8d53-4b0b-b350-9a486aaef277 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:39.772147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef9f2d-8d53-4b0b-b350-9a486aaef277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:39.773529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef9f2d-8d53-4b0b-b350-9a486aaef277 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:39.781579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 472/936] 10.222.0.79 () {56 vars in 1115 bytes} [Mon Jul 4 09:56:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:39.782197 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db7d3d-3ed0-451d-8bba-f7e36fca4323 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:39.782400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef9f2d-8d53-4b0b-b350-9a486aaef277 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:39.822581 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73520ecd-5eb1-48b3-a52e-4728f3f51a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:39.823562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73520ecd-5eb1-48b3-a52e-4728f3f51a06 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.824005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73520ecd-5eb1-48b3-a52e-4728f3f51a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.824516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73520ecd-5eb1-48b3-a52e-4728f3f51a06 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.825230 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73520ecd-5eb1-48b3-a52e-4728f3f51a06 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:39.825541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73520ecd-5eb1-48b3-a52e-4728f3f51a06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:39.827128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73520ecd-5eb1-48b3-a52e-4728f3f51a06 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:39.832242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 465/937] 10.222.0.79 () {56 vars in 1112 bytes} [Mon Jul 4 09:56:39 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:39.833034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73520ecd-5eb1-48b3-a52e-4728f3f51a06 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:39.911956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef9f2d-8d53-4b0b-b350-9a486aaef277 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:39.943661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:39.956456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73520ecd-5eb1-48b3-a52e-4728f3f51a06 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:39.961530 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093ca96-4b32-4846-b726-52d651a9ebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:39.962499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093ca96-4b32-4846-b726-52d651a9ebed None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:39.962949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093ca96-4b32-4846-b726-52d651a9ebed None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:39.963373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093ca96-4b32-4846-b726-52d651a9ebed None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:39.964876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093ca96-4b32-4846-b726-52d651a9ebed None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:39.965339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093ca96-4b32-4846-b726-52d651a9ebed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:39.967406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093ca96-4b32-4846-b726-52d651a9ebed None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:39.975085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 473/938] 10.222.0.79 () {56 vars in 1122 bytes} [Mon Jul 4 09:56:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:39.976497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:39.986426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093ca96-4b32-4846-b726-52d651a9ebed None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:40.009643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4de8ce-8687-4253-8f72-843f8750923b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:40.010507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de8ce-8687-4253-8f72-843f8750923b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:40.010793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de8ce-8687-4253-8f72-843f8750923b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:40.011069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de8ce-8687-4253-8f72-843f8750923b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:40.023923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4de8ce-8687-4253-8f72-843f8750923b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:40.025778 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4de8ce-8687-4253-8f72-843f8750923b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1d4de8ce-8687-4253-8f72-843f8750923b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:40.048454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:40.050370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:40.051879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:40.051879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:40.051879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:40.051879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:40.051879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 466/939] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:40.056488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4de8ce-8687-4253-8f72-843f8750923b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:40.108411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093ca96-4b32-4846-b726-52d651a9ebed None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:40.143388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f2315-35f5-41ae-af38-6a040e793065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:40.144045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f2315-35f5-41ae-af38-6a040e793065 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:40.144330 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f2315-35f5-41ae-af38-6a040e793065 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:40.144605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f2315-35f5-41ae-af38-6a040e793065 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:40.145494 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f2315-35f5-41ae-af38-6a040e793065 None admin] RBAC: Adding query filter params (interface=public, service_id=695b96afcb414be5900b96d7975b50a9, region_id=RegionOne) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:40.145753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f2315-35f5-41ae-af38-6a040e793065 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:40.147144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f2315-35f5-41ae-af38-6a040e793065 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:40.152785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 474/940] 10.222.0.79 () {56 vars in 1251 bytes} [Mon Jul 4 09:56:39 2022] GET /identity/v3/endpoints?service_id=695b96afcb414be5900b96d7975b50a9&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:40.152785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280a329-1c93-4832-b265-55cfcc9fb91b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:40.152785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280a329-1c93-4832-b265-55cfcc9fb91b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:40.152785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280a329-1c93-4832-b265-55cfcc9fb91b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:40.152785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 475/941] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:40.159760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280a329-1c93-4832-b265-55cfcc9fb91b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:40.191980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4de8ce-8687-4253-8f72-843f8750923b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:40.216824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:40.230135 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415aa04-a0a8-4710-98e8-b8bd7b368cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:40.230889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415aa04-a0a8-4710-98e8-b8bd7b368cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:40.231239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415aa04-a0a8-4710-98e8-b8bd7b368cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:40.231578 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415aa04-a0a8-4710-98e8-b8bd7b368cf0 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:40.232306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415aa04-a0a8-4710-98e8-b8bd7b368cf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:40.232616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415aa04-a0a8-4710-98e8-b8bd7b368cf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:40.234273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415aa04-a0a8-4710-98e8-b8bd7b368cf0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:40.246680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 467/942] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:40.247962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166c717-8f78-473e-be00-b9ef8b807228 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:40.248250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166c717-8f78-473e-be00-b9ef8b807228 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:40.248526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166c717-8f78-473e-be00-b9ef8b807228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:40.265674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1166c717-8f78-473e-be00-b9ef8b807228 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:40.267140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1166c717-8f78-473e-be00-b9ef8b807228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:40.274413 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280a329-1c93-4832-b265-55cfcc9fb91b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:40.292955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:40.310247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2e0a0-f32f-4be1-b3d3-1c4318b6d40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:40.310924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2e0a0-f32f-4be1-b3d3-1c4318b6d40e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:40.311212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2e0a0-f32f-4be1-b3d3-1c4318b6d40e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:40.311549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2e0a0-f32f-4be1-b3d3-1c4318b6d40e None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:40.312774 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2e0a0-f32f-4be1-b3d3-1c4318b6d40e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:40.314526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2e0a0-f32f-4be1-b3d3-1c4318b6d40e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:40.329659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 476/943] 10.222.0.79 () {56 vars in 1087 bytes} [Mon Jul 4 09:56:40 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:56:40.329659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2e0a0-f32f-4be1-b3d3-1c4318b6d40e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:40.354653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 468/944] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:40.377820 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166c717-8f78-473e-be00-b9ef8b807228 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:40.455468 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2e0a0-f32f-4be1-b3d3-1c4318b6d40e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:40.478022 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:40.485555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166c717-8f78-473e-be00-b9ef8b807228 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:40.501466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9d834-a02f-4f2c-8105-0f808a907b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:40.502323 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9d834-a02f-4f2c-8105-0f808a907b26 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:40.502612 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9d834-a02f-4f2c-8105-0f808a907b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:40.502890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9d834-a02f-4f2c-8105-0f808a907b26 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:40.510928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9d834-a02f-4f2c-8105-0f808a907b26 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:40.512394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9d834-a02f-4f2c-8105-0f808a907b26 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:40.530470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1aff20-b7f2-45d8-8344-4935cc8e60e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-07c9d834-a02f-4f2c-8105-0f808a907b26 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:40.530948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:40.531882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:40.533746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:40.533746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:40.533746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:40.533746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:40.533746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:40.533746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:40.533746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:40.533746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:40.533746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:56:40.533746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:40.534384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1aff20-b7f2-45d8-8344-4935cc8e60e5 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:40.534384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1aff20-b7f2-45d8-8344-4935cc8e60e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:40.534384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1aff20-b7f2-45d8-8344-4935cc8e60e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:40.534613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 477/945] 10.222.0.79 () {56 vars in 1117 bytes} [Mon Jul 4 09:56:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:40.534703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1aff20-b7f2-45d8-8344-4935cc8e60e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:40.534703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1aff20-b7f2-45d8-8344-4935cc8e60e5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:40.535996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: WARNING keystone.server.flask.common [None req-2a1aff20-b7f2-45d8-8344-4935cc8e60e5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 04 09:56:40.538415 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9d834-a02f-4f2c-8105-0f808a907b26 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:40.561739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.sql.core [None req-2a1aff20-b7f2-45d8-8344-4935cc8e60e5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 09:56:40.561739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 04 09:56:40.561739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [parameters: {'id': '1b00c5019ddc499e96a6f1dd8f1c2be2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 04 09:56:40.561739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80099) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 09:56:40.569318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-2a1aff20-b7f2-45d8-8344-4935cc8e60e5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b00c5019ddc499e96a6f1dd8f1c2be2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b00c5019ddc499e96a6f1dd8f1c2be2. Jul 04 09:56:40.569318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:40.569318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:56:40.569318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:56:40.569318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:56:40.569318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:56:40.569318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:56:40.569318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:56:40.569318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:56:40.569318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:56:40.569318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:56:40.569318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:56:40.569318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result.read() Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 04 09:56:40.570133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application yield resource Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise self.value Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application session.commit() Jul 04 09:56:40.571389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.session.flush() Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application compat.raise_( Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 09:56:40.572837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 09:56:40.574065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result.read() Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:56:40.575027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [parameters: {'id': '1b00c5019ddc499e96a6f1dd8f1c2be2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:40.576887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:40.579268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:40.581395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:40.581395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 04 09:56:40.581395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 04 09:56:40.581395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:40.581395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:40.581395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 04 09:56:40.581395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 04 09:56:40.581395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b00c5019ddc499e96a6f1dd8f1c2be2. Jul 04 09:56:40.581395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:40.581395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 469/946] 10.222.0.79 () {60 vars in 1140 bytes} [Mon Jul 4 09:56:40 2022] POST /identity/v3/projects => generated 262 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:40.581395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1aff20-b7f2-45d8-8344-4935cc8e60e5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:40.598191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:40.690866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9d834-a02f-4f2c-8105-0f808a907b26 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:40.708840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1aff20-b7f2-45d8-8344-4935cc8e60e5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:40.736440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad14192-7aca-4496-a330-39e23ef4673d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:40.738122 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad14192-7aca-4496-a330-39e23ef4673d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:40.738613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad14192-7aca-4496-a330-39e23ef4673d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:40.739110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad14192-7aca-4496-a330-39e23ef4673d None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:40.740802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad14192-7aca-4496-a330-39e23ef4673d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:40.741529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad14192-7aca-4496-a330-39e23ef4673d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:40.743268 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad14192-7aca-4496-a330-39e23ef4673d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:40.745231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbccfdd-b5a7-4e42-bd9e-a0e698ba2bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:40.746165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbccfdd-b5a7-4e42-bd9e-a0e698ba2bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:40.746512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbccfdd-b5a7-4e42-bd9e-a0e698ba2bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:40.746856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbccfdd-b5a7-4e42-bd9e-a0e698ba2bf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:40.754125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbccfdd-b5a7-4e42-bd9e-a0e698ba2bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:40.756363 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbccfdd-b5a7-4e42-bd9e-a0e698ba2bf2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:40.765688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 478/947] 10.222.0.79 () {56 vars in 1149 bytes} [Mon Jul 4 09:56:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-7bbccfdd-b5a7-4e42-bd9e-a0e698ba2bf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:40.770751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:40.771645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:40.772525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:40.772525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:40.772525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:40.772525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:40.772525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:40.772525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:40.772525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:40.772525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:40.774563 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad14192-7aca-4496-a330-39e23ef4673d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:40.776644 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 470/948] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:40.777580 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbccfdd-b5a7-4e42-bd9e-a0e698ba2bf2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:40.811114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:40.880118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:40.898842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad14192-7aca-4496-a330-39e23ef4673d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:40.910586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbccfdd-b5a7-4e42-bd9e-a0e698ba2bf2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:40.942150 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70e4ed-063e-4758-b92d-602ab7a306f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:40.942979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70e4ed-063e-4758-b92d-602ab7a306f9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:40.943337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70e4ed-063e-4758-b92d-602ab7a306f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:40.943834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70e4ed-063e-4758-b92d-602ab7a306f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:40.953504 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfda0e-1b41-4066-ab9c-ff39ef6ca4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:40.954870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfda0e-1b41-4066-ab9c-ff39ef6ca4ae None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:40.955440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfda0e-1b41-4066-ab9c-ff39ef6ca4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:40.955970 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfda0e-1b41-4066-ab9c-ff39ef6ca4ae None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:40.956451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70e4ed-063e-4758-b92d-602ab7a306f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:40.957043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbfda0e-1b41-4066-ab9c-ff39ef6ca4ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:40.957501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbfda0e-1b41-4066-ab9c-ff39ef6ca4ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:40.958234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70e4ed-063e-4758-b92d-602ab7a306f9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:40.959301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbfda0e-1b41-4066-ab9c-ff39ef6ca4ae None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-ca70e4ed-063e-4758-b92d-602ab7a306f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:40.979005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:40.980130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:40.981250 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:40.981250 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:40.981250 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:40.981250 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:40.982164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 479/949] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:40.984937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 471/950] 10.222.0.79 () {56 vars in 1112 bytes} [Mon Jul 4 09:56:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:40.990820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70e4ed-063e-4758-b92d-602ab7a306f9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:41.086891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:41.105992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70e4ed-063e-4758-b92d-602ab7a306f9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:41.145918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d5ec7-8218-43cd-9cce-a8c888988024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:41.146617 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d5ec7-8218-43cd-9cce-a8c888988024 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:41.146878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d5ec7-8218-43cd-9cce-a8c888988024 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:41.147153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d5ec7-8218-43cd-9cce-a8c888988024 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:41.147886 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d5ec7-8218-43cd-9cce-a8c888988024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:41.148191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d5ec7-8218-43cd-9cce-a8c888988024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:41.149565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d5ec7-8218-43cd-9cce-a8c888988024 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:41.165876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 480/951] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:41.167741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfda0e-1b41-4066-ab9c-ff39ef6ca4ae None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:41.276295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfda0e-1b41-4066-ab9c-ff39ef6ca4ae None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:41.311140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380df46f-8934-4ff6-b9ed-c814e2aa25cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:41.311978 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380df46f-8934-4ff6-b9ed-c814e2aa25cc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:41.312266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380df46f-8934-4ff6-b9ed-c814e2aa25cc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:41.312543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380df46f-8934-4ff6-b9ed-c814e2aa25cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:41.315621 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:41.320319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380df46f-8934-4ff6-b9ed-c814e2aa25cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:41.321818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380df46f-8934-4ff6-b9ed-c814e2aa25cc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-380df46f-8934-4ff6-b9ed-c814e2aa25cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:41.336741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:41.337664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:41.338572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:41.338572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:41.338572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:41.338572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:41.338572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:41.338572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:41.338572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:41.338572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:41.338572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 472/952] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:41.344082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d5ec7-8218-43cd-9cce-a8c888988024 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:41.445995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d5ec7-8218-43cd-9cce-a8c888988024 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:41.480453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0741486-9801-409e-adfd-71c3117a8af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:41.481117 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0741486-9801-409e-adfd-71c3117a8af3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:41.481400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0741486-9801-409e-adfd-71c3117a8af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:41.481703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0741486-9801-409e-adfd-71c3117a8af3 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:41.482494 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0741486-9801-409e-adfd-71c3117a8af3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:41.482796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0741486-9801-409e-adfd-71c3117a8af3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:41.484354 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0741486-9801-409e-adfd-71c3117a8af3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:41.497364 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 481/953] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:41.503579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380df46f-8934-4ff6-b9ed-c814e2aa25cc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:41.612513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380df46f-8934-4ff6-b9ed-c814e2aa25cc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:41.647490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6295982-232e-48a4-91f2-5684dfbbb676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:41.648391 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6295982-232e-48a4-91f2-5684dfbbb676 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:41.648675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6295982-232e-48a4-91f2-5684dfbbb676 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:41.648952 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6295982-232e-48a4-91f2-5684dfbbb676 None admin] PATH_INFO: `/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/5bdf239fa2bc4f1b9e484f916c438c1a/roles/44488839b54a4c9197dca8c9e2226e84` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:41.651278 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6295982-232e-48a4-91f2-5684dfbbb676 None admin] RBAC: Authorizing `identity:create_grant(project_id=f32306262e8744eab4bdc6d28e24e6d0, user_id=5bdf239fa2bc4f1b9e484f916c438c1a, role_id=44488839b54a4c9197dca8c9e2226e84)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:41.652841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6295982-232e-48a4-91f2-5684dfbbb676 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:41.670755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 473/954] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:56:41 2022] PUT /identity/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/5bdf239fa2bc4f1b9e484f916c438c1a/roles/44488839b54a4c9197dca8c9e2226e84 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:56:42.002640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f5deb-0b63-4cd0-b7ea-318fc3af5336 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:42.003308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f5deb-0b63-4cd0-b7ea-318fc3af5336 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:42.003793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f5deb-0b63-4cd0-b7ea-318fc3af5336 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:42.005649 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 482/955] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:42.011175 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c37caf-2712-4d3e-b23f-9207bfa011b5 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:42.011685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c37caf-2712-4d3e-b23f-9207bfa011b5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:42.012171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c37caf-2712-4d3e-b23f-9207bfa011b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:42.030164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a4c37caf-2712-4d3e-b23f-9207bfa011b5 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:42.032145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a4c37caf-2712-4d3e-b23f-9207bfa011b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:42.089651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:42.109145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 474/956] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:42.142949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f5deb-0b63-4cd0-b7ea-318fc3af5336 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:42.147016 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:42.247018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f5deb-0b63-4cd0-b7ea-318fc3af5336 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:42.283927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf07a39-b57f-4d24-becf-d4784a0c18fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:42.284601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf07a39-b57f-4d24-becf-d4784a0c18fc None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:42.284884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf07a39-b57f-4d24-becf-d4784a0c18fc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:42.285155 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf07a39-b57f-4d24-becf-d4784a0c18fc None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:42.285918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf07a39-b57f-4d24-becf-d4784a0c18fc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:42.287292 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf07a39-b57f-4d24-becf-d4784a0c18fc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:42.305016 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 483/957] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:56:42 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:56:42.449057 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2c537-18ce-4e93-acb4-5e4659c62d0f None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:42.449783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2c537-18ce-4e93-acb4-5e4659c62d0f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:42.449783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2c537-18ce-4e93-acb4-5e4659c62d0f None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:42.451261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 475/958] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:42.457781 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0cb96-809b-4a6e-b574-9e95a7f54a54 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:42.458064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0cb96-809b-4a6e-b574-9e95a7f54a54 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:42.458336 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0cb96-809b-4a6e-b574-9e95a7f54a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:42.475968 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-70c0cb96-809b-4a6e-b574-9e95a7f54a54 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:42.477170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-70c0cb96-809b-4a6e-b574-9e95a7f54a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:42.536805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:42.556860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 484/959] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:42.577393 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2c537-18ce-4e93-acb4-5e4659c62d0f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:42.681979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2c537-18ce-4e93-acb4-5e4659c62d0f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:42.722027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82a9ba-edf1-49aa-b5b2-bf51ed12faa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:42.722862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82a9ba-edf1-49aa-b5b2-bf51ed12faa2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:42.723139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82a9ba-edf1-49aa-b5b2-bf51ed12faa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:42.723412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82a9ba-edf1-49aa-b5b2-bf51ed12faa2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:42.724103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82a9ba-edf1-49aa-b5b2-bf51ed12faa2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:42.725871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82a9ba-edf1-49aa-b5b2-bf51ed12faa2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-9e82a9ba-edf1-49aa-b5b2-bf51ed12faa2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:42.744460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:42.745629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:42.746545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:42.749888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 476/960] 10.222.0.79 () {56 vars in 1126 bytes} [Mon Jul 4 09:56:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:42.753046 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0cb96-809b-4a6e-b574-9e95a7f54a54 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:42.865605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0cb96-809b-4a6e-b574-9e95a7f54a54 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:42.900084 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cede22-0057-495f-8238-aa97f5863222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:42.901139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cede22-0057-495f-8238-aa97f5863222 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:42.901729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cede22-0057-495f-8238-aa97f5863222 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:42.902219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cede22-0057-495f-8238-aa97f5863222 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:42.903110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cede22-0057-495f-8238-aa97f5863222 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:42.903611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cede22-0057-495f-8238-aa97f5863222 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:42.905219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cede22-0057-495f-8238-aa97f5863222 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:42.910953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 485/961] 10.222.0.79 () {56 vars in 1122 bytes} [Mon Jul 4 09:56:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:42.919004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82a9ba-edf1-49aa-b5b2-bf51ed12faa2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:42.971405 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:43.030014 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82a9ba-edf1-49aa-b5b2-bf51ed12faa2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:43.064372 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002a622-7b2d-41c8-bf6c-2fd54d582a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:43.065045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002a622-7b2d-41c8-bf6c-2fd54d582a5b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:43.065334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002a622-7b2d-41c8-bf6c-2fd54d582a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:43.065603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002a622-7b2d-41c8-bf6c-2fd54d582a5b None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:43.066334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002a622-7b2d-41c8-bf6c-2fd54d582a5b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:43.066643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002a622-7b2d-41c8-bf6c-2fd54d582a5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:43.068037 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002a622-7b2d-41c8-bf6c-2fd54d582a5b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:43.073721 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 477/962] 10.222.0.79 () {56 vars in 1122 bytes} [Mon Jul 4 09:56:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:43.079667 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cede22-0057-495f-8238-aa97f5863222 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:43.141660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:43.185421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cede22-0057-495f-8238-aa97f5863222 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:43.219927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb885310-9869-41a8-97e7-194ebe89fdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:43.220969 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb885310-9869-41a8-97e7-194ebe89fdfc None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:43.221256 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb885310-9869-41a8-97e7-194ebe89fdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:43.221530 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb885310-9869-41a8-97e7-194ebe89fdfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:43.222331 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb885310-9869-41a8-97e7-194ebe89fdfc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:43.223715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb885310-9869-41a8-97e7-194ebe89fdfc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:43.253862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 486/963] 10.222.0.79 () {60 vars in 1144 bytes} [Mon Jul 4 09:56:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:56:43.256779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c2861-d526-4f51-aa24-4184744885e6 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:43.257073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c2861-d526-4f51-aa24-4184744885e6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:43.257408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c2861-d526-4f51-aa24-4184744885e6 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:43.259236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 478/964] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:56:43.265522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7306c-b3fe-4a5a-bd1c-85ae8457e984 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:43.265876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7306c-b3fe-4a5a-bd1c-85ae8457e984 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:43.266192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7306c-b3fe-4a5a-bd1c-85ae8457e984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:43.285279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-15b7306c-b3fe-4a5a-bd1c-85ae8457e984 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:43.286534 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-15b7306c-b3fe-4a5a-bd1c-85ae8457e984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:43.372858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 487/965] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:43.398323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c2861-d526-4f51-aa24-4184744885e6 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:43.510745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c2861-d526-4f51-aa24-4184744885e6 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:43.527356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:43.549309 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207279e-58f7-45fa-bbad-2427e132fcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:43.550025 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207279e-58f7-45fa-bbad-2427e132fcae None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:43.550310 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207279e-58f7-45fa-bbad-2427e132fcae None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:43.550589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207279e-58f7-45fa-bbad-2427e132fcae None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:43.551296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207279e-58f7-45fa-bbad-2427e132fcae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:43.552763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207279e-58f7-45fa-bbad-2427e132fcae None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:43.571116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.sql.core [None req-8207279e-58f7-45fa-bbad-2427e132fcae None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 09:56:43.571116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 04 09:56:43.571116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [parameters: {'user_id': 'ec0e173baa3c4a5db0393553e3ba14f2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 04 09:56:43.571116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80099) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8207279e-58f7-45fa-bbad-2427e132fcae None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result.read() Jul 04 09:56:43.579882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:56:43.581959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application yield resource Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise self.value Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application session.commit() Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 09:56:43.584145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.session.flush() Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application compat.raise_( Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 09:56:43.586273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:56:43.588338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result.read() Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:56:43.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ec0e173baa3c4a5db0393553e3ba14f2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:43.598714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 04 09:56:43.601053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 04 09:56:43.603191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 04 09:56:43.603191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 04 09:56:43.603191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 09:56:43.603191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 09:56:43.603191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 04 09:56:43.603191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:43.603191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 479/966] 10.222.0.79 () {60 vars in 1131 bytes} [Mon Jul 4 09:56:43 2022] POST /identity/v3/users => generated 160 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:43.604074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7306c-b3fe-4a5a-bd1c-85ae8457e984 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:43.630062 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:43.702165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7306c-b3fe-4a5a-bd1c-85ae8457e984 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:43.738495 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937a378-4511-478b-a625-82c3c7e2d1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:43.739361 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937a378-4511-478b-a625-82c3c7e2d1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:43.739652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937a378-4511-478b-a625-82c3c7e2d1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:43.739930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937a378-4511-478b-a625-82c3c7e2d1e6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:43.748013 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937a378-4511-478b-a625-82c3c7e2d1e6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:43.749689 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937a378-4511-478b-a625-82c3c7e2d1e6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:43.752262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d1d0e-39fd-48c5-9275-2ab0dd147b18 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:43.752553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d1d0e-39fd-48c5-9275-2ab0dd147b18 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:43.752828 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d1d0e-39fd-48c5-9275-2ab0dd147b18 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:43.761839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 480/967] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:56:43.763013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea91f47-da62-4470-8f59-2c556a4e7959 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:43.763296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea91f47-da62-4470-8f59-2c556a4e7959 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:43.763572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea91f47-da62-4470-8f59-2c556a4e7959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-0937a378-4511-478b-a625-82c3c7e2d1e6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:43.765233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:43.767401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:43.769602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:43.769602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:43.769602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:43.769602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:43.769602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:43.769602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:43.769602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:43.769602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:43.769602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:56:43.769602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:43.769602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 488/968] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:56:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:43.777660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937a378-4511-478b-a625-82c3c7e2d1e6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:43.781180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6ea91f47-da62-4470-8f59-2c556a4e7959 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:43.782745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6ea91f47-da62-4470-8f59-2c556a4e7959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:43.856691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 481/969] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:43.880191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea91f47-da62-4470-8f59-2c556a4e7959 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:43.882782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:43.888023 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937a378-4511-478b-a625-82c3c7e2d1e6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:43.934424 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290ede8-0ae7-4151-b016-5b8261b4e41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:43.935616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290ede8-0ae7-4151-b016-5b8261b4e41c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:43.935898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290ede8-0ae7-4151-b016-5b8261b4e41c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:43.936400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290ede8-0ae7-4151-b016-5b8261b4e41c None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:43.937326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290ede8-0ae7-4151-b016-5b8261b4e41c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:43.937645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290ede8-0ae7-4151-b016-5b8261b4e41c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:43.939766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290ede8-0ae7-4151-b016-5b8261b4e41c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:43.986559 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 489/970] 10.222.0.79 () {56 vars in 1101 bytes} [Mon Jul 4 09:56:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:43.986745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea91f47-da62-4470-8f59-2c556a4e7959 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:44.005341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:44.038144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f4ac4-47c5-4606-82e2-d7bfb5848f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:44.039125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f4ac4-47c5-4606-82e2-d7bfb5848f2f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:44.039731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f4ac4-47c5-4606-82e2-d7bfb5848f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:44.040010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f4ac4-47c5-4606-82e2-d7bfb5848f2f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:44.052848 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f4ac4-47c5-4606-82e2-d7bfb5848f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-440f4ac4-47c5-4606-82e2-d7bfb5848f2f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:44.061306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:44.062456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:44.063300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:44.063300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:44.063300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:44.063300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:44.063300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:44.063300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:44.063300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:44.063300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:44.063300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:44.063300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:44.063300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:56:44.063300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:44.063300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 482/971] 10.222.0.79 () {56 vars in 1120 bytes} [Mon Jul 4 09:56:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:44.068912 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290ede8-0ae7-4151-b016-5b8261b4e41c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:44.194972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290ede8-0ae7-4151-b016-5b8261b4e41c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:44.230989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cdc36-17f8-4e0c-851a-e8c62f78c4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:44.231663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cdc36-17f8-4e0c-851a-e8c62f78c4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:44.231943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cdc36-17f8-4e0c-851a-e8c62f78c4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:44.232215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cdc36-17f8-4e0c-851a-e8c62f78c4e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:44.232986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cdc36-17f8-4e0c-851a-e8c62f78c4e1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:44.233304 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cdc36-17f8-4e0c-851a-e8c62f78c4e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:44.234793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cdc36-17f8-4e0c-851a-e8c62f78c4e1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:44.244050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 490/972] 10.222.0.79 () {56 vars in 1115 bytes} [Mon Jul 4 09:56:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:44.250268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f4ac4-47c5-4606-82e2-d7bfb5848f2f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:44.355679 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f4ac4-47c5-4606-82e2-d7bfb5848f2f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:44.389689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e525c18-8349-4243-96eb-63d36a7d393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:44.390501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e525c18-8349-4243-96eb-63d36a7d393b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:44.390793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e525c18-8349-4243-96eb-63d36a7d393b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:44.391064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e525c18-8349-4243-96eb-63d36a7d393b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:44.402486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e525c18-8349-4243-96eb-63d36a7d393b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:44.404117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e525c18-8349-4243-96eb-63d36a7d393b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-0e525c18-8349-4243-96eb-63d36a7d393b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:44.422300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:44.423248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:44.424138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:44.424138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:44.424138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:44.424138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:44.424138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 483/973] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:44.431007 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cdc36-17f8-4e0c-851a-e8c62f78c4e1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:44.451459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:44.492706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72729b-7645-4970-8c84-2867510788ab None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:44.492970 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72729b-7645-4970-8c84-2867510788ab None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:44.493260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72729b-7645-4970-8c84-2867510788ab None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:44.497876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 484/974] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:56:44.500910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ed079-a630-448b-b0bd-8058788f9c2c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:44.501323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ed079-a630-448b-b0bd-8058788f9c2c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:44.501583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ed079-a630-448b-b0bd-8058788f9c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:44.521238 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4a2ed079-a630-448b-b0bd-8058788f9c2c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:44.522482 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4a2ed079-a630-448b-b0bd-8058788f9c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:44.538675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:44.550749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cdc36-17f8-4e0c-851a-e8c62f78c4e1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:44.587446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe43312-8c71-4f53-9ea0-87af61b54aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:44.588141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe43312-8c71-4f53-9ea0-87af61b54aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:44.588410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe43312-8c71-4f53-9ea0-87af61b54aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:44.588696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe43312-8c71-4f53-9ea0-87af61b54aa9 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:44.589418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe43312-8c71-4f53-9ea0-87af61b54aa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:44.589764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe43312-8c71-4f53-9ea0-87af61b54aa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:44.591150 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe43312-8c71-4f53-9ea0-87af61b54aa9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:44.601146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 485/975] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:44.608912 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 491/976] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:44.614584 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ed079-a630-448b-b0bd-8058788f9c2c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:44.625554 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe43312-8c71-4f53-9ea0-87af61b54aa9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:44.710909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:44.714829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ed079-a630-448b-b0bd-8058788f9c2c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:44.736763 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe43312-8c71-4f53-9ea0-87af61b54aa9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:44.749246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216127ad-97ec-4b56-af54-610b89bdc107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:44.750194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216127ad-97ec-4b56-af54-610b89bdc107 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:44.750473 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216127ad-97ec-4b56-af54-610b89bdc107 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:44.750743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216127ad-97ec-4b56-af54-610b89bdc107 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:44.759152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216127ad-97ec-4b56-af54-610b89bdc107 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:44.760610 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216127ad-97ec-4b56-af54-610b89bdc107 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:44.772599 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d30ad-b9a5-46d4-9db9-29c675b15828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:44.773605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d30ad-b9a5-46d4-9db9-29c675b15828 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:44.774057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d30ad-b9a5-46d4-9db9-29c675b15828 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:44.774478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d30ad-b9a5-46d4-9db9-29c675b15828 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:44.775272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d30ad-b9a5-46d4-9db9-29c675b15828 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-216127ad-97ec-4b56-af54-610b89bdc107 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:44.775930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:44.777076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:44.778136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:44.778136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:44.778136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:44.778136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:44.778136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:44.778136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:44.778136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:44.778136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:44.778136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:56:44.778136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:44.778136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 486/977] 10.222.0.79 () {56 vars in 1117 bytes} [Mon Jul 4 09:56:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:44.778771 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d30ad-b9a5-46d4-9db9-29c675b15828 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:44.782855 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216127ad-97ec-4b56-af54-610b89bdc107 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-eb5d30ad-b9a5-46d4-9db9-29c675b15828 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:44.792042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:44.793067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:56:44.794930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:44.795726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:44.795827 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 492/978] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:44.800507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d30ad-b9a5-46d4-9db9-29c675b15828 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:44.905031 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216127ad-97ec-4b56-af54-610b89bdc107 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:44.907640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d30ad-b9a5-46d4-9db9-29c675b15828 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:44.940889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012833d-8ca4-496d-b25f-1806d1626f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:44.941942 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c18b07-a210-4a77-9a2d-b89910a11b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:44.942089 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012833d-8ca4-496d-b25f-1806d1626f05 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:44.942551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012833d-8ca4-496d-b25f-1806d1626f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:44.942654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c18b07-a210-4a77-9a2d-b89910a11b00 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:44.942884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c18b07-a210-4a77-9a2d-b89910a11b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:44.943163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c18b07-a210-4a77-9a2d-b89910a11b00 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:44.943475 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012833d-8ca4-496d-b25f-1806d1626f05 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:44.943869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c18b07-a210-4a77-9a2d-b89910a11b00 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:44.944176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c18b07-a210-4a77-9a2d-b89910a11b00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:44.944840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012833d-8ca4-496d-b25f-1806d1626f05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:44.945314 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012833d-8ca4-496d-b25f-1806d1626f05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:44.945552 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c18b07-a210-4a77-9a2d-b89910a11b00 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:44.947233 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012833d-8ca4-496d-b25f-1806d1626f05 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:44.950761 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 493/979] 10.222.0.79 () {56 vars in 1112 bytes} [Mon Jul 4 09:56:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:44.956782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c18b07-a210-4a77-9a2d-b89910a11b00 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:44.961025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:44.971384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 487/980] 10.222.0.79 () {56 vars in 1149 bytes} [Mon Jul 4 09:56:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:44.972562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012833d-8ca4-496d-b25f-1806d1626f05 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:45.041774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:45.066152 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c18b07-a210-4a77-9a2d-b89910a11b00 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:45.078106 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012833d-8ca4-496d-b25f-1806d1626f05 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:45.100815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95319c9-3643-48c0-8bc8-595addb18d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:45.101602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95319c9-3643-48c0-8bc8-595addb18d5c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:45.102216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95319c9-3643-48c0-8bc8-595addb18d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:45.102324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95319c9-3643-48c0-8bc8-595addb18d5c None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:45.102993 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95319c9-3643-48c0-8bc8-595addb18d5c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:45.103300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95319c9-3643-48c0-8bc8-595addb18d5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:45.104884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95319c9-3643-48c0-8bc8-595addb18d5c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:45.110588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 494/981] 10.222.0.79 () {56 vars in 1112 bytes} [Mon Jul 4 09:56:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:45.116162 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95319c9-3643-48c0-8bc8-595addb18d5c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:45.117482 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97244d1-8c82-4fee-8fad-090239a70c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:45.118247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97244d1-8c82-4fee-8fad-090239a70c03 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:45.118526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97244d1-8c82-4fee-8fad-090239a70c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:45.118805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97244d1-8c82-4fee-8fad-090239a70c03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:45.186730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:45.195015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97244d1-8c82-4fee-8fad-090239a70c03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:45.196493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97244d1-8c82-4fee-8fad-090239a70c03 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:45.224128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95319c9-3643-48c0-8bc8-595addb18d5c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:45.259724 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bf5a8-5cae-42b2-bfae-978af3b0829d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:45.260385 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bf5a8-5cae-42b2-bfae-978af3b0829d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:45.260674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bf5a8-5cae-42b2-bfae-978af3b0829d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:45.260948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bf5a8-5cae-42b2-bfae-978af3b0829d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:45.261767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bf5a8-5cae-42b2-bfae-978af3b0829d None admin] RBAC: Adding query filter params (interface=public, service_id=cd8c5e79034e47a0ba54b54eb6d339c0, region_id=RegionOne) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:45.262082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bf5a8-5cae-42b2-bfae-978af3b0829d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:45.263462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bf5a8-5cae-42b2-bfae-978af3b0829d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:45.263841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:45.269663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 495/982] 10.222.0.79 () {56 vars in 1251 bytes} [Mon Jul 4 09:56:45 2022] GET /identity/v3/endpoints?service_id=cd8c5e79034e47a0ba54b54eb6d339c0&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:45.274595 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bf5a8-5cae-42b2-bfae-978af3b0829d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:45.305882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 488/983] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:44 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:56:45.311664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97244d1-8c82-4fee-8fad-090239a70c03 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:45.380881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bf5a8-5cae-42b2-bfae-978af3b0829d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:45.412924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:45.417701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97244d1-8c82-4fee-8fad-090239a70c03 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:45.418207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5512d37-9f79-448f-b135-89712381b4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:45.418879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5512d37-9f79-448f-b135-89712381b4ab None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:45.419153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5512d37-9f79-448f-b135-89712381b4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:45.419428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5512d37-9f79-448f-b135-89712381b4ab None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:45.420110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5512d37-9f79-448f-b135-89712381b4ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:45.421524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5512d37-9f79-448f-b135-89712381b4ab None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:45.427798 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 496/984] 10.222.0.79 () {56 vars in 1087 bytes} [Mon Jul 4 09:56:45 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:56:45.429065 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac995-69b6-4e4a-8e3a-e52ba5b2d096 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:45.429356 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac995-69b6-4e4a-8e3a-e52ba5b2d096 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:45.429686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac995-69b6-4e4a-8e3a-e52ba5b2d096 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:45.430874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 497/985] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:45.436883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efa6af-8ac0-4cad-8a08-e600dcf7f532 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:45.437359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efa6af-8ac0-4cad-8a08-e600dcf7f532 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:45.437800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efa6af-8ac0-4cad-8a08-e600dcf7f532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:45.455680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a7efa6af-8ac0-4cad-8a08-e600dcf7f532 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:45.456916 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a7efa6af-8ac0-4cad-8a08-e600dcf7f532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:45.464661 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd58733-29c9-438d-a466-bc683b8511fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:45.465488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd58733-29c9-438d-a466-bc683b8511fe None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:45.465806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd58733-29c9-438d-a466-bc683b8511fe None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:45.466079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd58733-29c9-438d-a466-bc683b8511fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:45.478071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd58733-29c9-438d-a466-bc683b8511fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:45.479648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd58733-29c9-438d-a466-bc683b8511fe None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8fd58733-29c9-438d-a466-bc683b8511fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:45.498775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:45.499790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:45.501151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:45.501151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:45.501151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:45.501151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:45.501151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 489/986] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:45.506089 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd58733-29c9-438d-a466-bc683b8511fe None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:45.526969 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:45.541217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 498/987] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:45.564596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efa6af-8ac0-4cad-8a08-e600dcf7f532 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:45.628934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd58733-29c9-438d-a466-bc683b8511fe None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:45.667086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5fe6b-b205-4f28-bafe-20502dd9fabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:45.667765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5fe6b-b205-4f28-bafe-20502dd9fabc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:45.668048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5fe6b-b205-4f28-bafe-20502dd9fabc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:45.668324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5fe6b-b205-4f28-bafe-20502dd9fabc None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:45.669051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5fe6b-b205-4f28-bafe-20502dd9fabc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:45.669547 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5fe6b-b205-4f28-bafe-20502dd9fabc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:45.671402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5fe6b-b205-4f28-bafe-20502dd9fabc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:45.678442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efa6af-8ac0-4cad-8a08-e600dcf7f532 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:45.684793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 490/988] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:45.690737 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5fe6b-b205-4f28-bafe-20502dd9fabc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:45.713128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:45.724358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3ff0f-6184-4931-ac8b-5936114e21ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:45.725710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3ff0f-6184-4931-ac8b-5936114e21ba None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:45.726160 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3ff0f-6184-4931-ac8b-5936114e21ba None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:45.726334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3ff0f-6184-4931-ac8b-5936114e21ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:45.726988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3ff0f-6184-4931-ac8b-5936114e21ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:45.728369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3ff0f-6184-4931-ac8b-5936114e21ba None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-17c3ff0f-6184-4931-ac8b-5936114e21ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:45.742058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:45.743074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:45.743933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 499/989] 10.222.0.79 () {56 vars in 1126 bytes} [Mon Jul 4 09:56:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:45.749662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3ff0f-6184-4931-ac8b-5936114e21ba None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:45.775906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:45.799167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5fe6b-b205-4f28-bafe-20502dd9fabc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:45.838688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c16913-560d-4233-8495-19a129d8a88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:45.839385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c16913-560d-4233-8495-19a129d8a88c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:45.839697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c16913-560d-4233-8495-19a129d8a88c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:45.839974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c16913-560d-4233-8495-19a129d8a88c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:45.848148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c16913-560d-4233-8495-19a129d8a88c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:45.849896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c16913-560d-4233-8495-19a129d8a88c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:45.863269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3ff0f-6184-4931-ac8b-5936114e21ba None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-76c16913-560d-4233-8495-19a129d8a88c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:45.871408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:45.872524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:45.879890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:45.879890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:45.879890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:45.879890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:45.879890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:45.879890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:45.879890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:45.879890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:45.879890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 491/990] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:45.880390 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c16913-560d-4233-8495-19a129d8a88c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:45.898724 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f77ed-f94a-4913-9f3c-5ffcf6830a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:45.899416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f77ed-f94a-4913-9f3c-5ffcf6830a20 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:45.899706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f77ed-f94a-4913-9f3c-5ffcf6830a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:45.899968 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f77ed-f94a-4913-9f3c-5ffcf6830a20 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:45.900669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f77ed-f94a-4913-9f3c-5ffcf6830a20 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:45.900959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f77ed-f94a-4913-9f3c-5ffcf6830a20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:45.902373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f77ed-f94a-4913-9f3c-5ffcf6830a20 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:45.907780 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 500/991] 10.222.0.79 () {56 vars in 1122 bytes} [Mon Jul 4 09:56:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:45.913156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f77ed-f94a-4913-9f3c-5ffcf6830a20 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:45.970319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:45.990038 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c16913-560d-4233-8495-19a129d8a88c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:46.017293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f77ed-f94a-4913-9f3c-5ffcf6830a20 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:46.030125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e5bed-4e85-4a72-ac51-6277f524b4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:46.030981 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e5bed-4e85-4a72-ac51-6277f524b4fc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:46.031199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e5bed-4e85-4a72-ac51-6277f524b4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:46.031472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e5bed-4e85-4a72-ac51-6277f524b4fc None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:46.032252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e5bed-4e85-4a72-ac51-6277f524b4fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:46.032560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e5bed-4e85-4a72-ac51-6277f524b4fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:46.034024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e5bed-4e85-4a72-ac51-6277f524b4fc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:46.040924 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:46.052066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 492/992] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:46.052790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e5bed-4e85-4a72-ac51-6277f524b4fc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:46.054963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c5d5e-89f4-4b3a-8b48-554cd3a7d011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:46.055826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c5d5e-89f4-4b3a-8b48-554cd3a7d011 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:46.056109 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c5d5e-89f4-4b3a-8b48-554cd3a7d011 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:46.057218 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c5d5e-89f4-4b3a-8b48-554cd3a7d011 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:46.058111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c5d5e-89f4-4b3a-8b48-554cd3a7d011 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:46.058429 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c5d5e-89f4-4b3a-8b48-554cd3a7d011 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:46.059814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c5d5e-89f4-4b3a-8b48-554cd3a7d011 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:46.067251 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 501/993] 10.222.0.79 () {56 vars in 1122 bytes} [Mon Jul 4 09:56:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:46.072175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c5d5e-89f4-4b3a-8b48-554cd3a7d011 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:46.176852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e5bed-4e85-4a72-ac51-6277f524b4fc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:46.188804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c5d5e-89f4-4b3a-8b48-554cd3a7d011 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:46.206574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:46.214060 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c89b7-508e-4e31-94fa-c1c6fd0ec24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:46.214730 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c89b7-508e-4e31-94fa-c1c6fd0ec24a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:46.215025 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c89b7-508e-4e31-94fa-c1c6fd0ec24a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:46.215306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c89b7-508e-4e31-94fa-c1c6fd0ec24a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:46.216167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c89b7-508e-4e31-94fa-c1c6fd0ec24a None admin] RBAC: Adding query filter params (role.id=44488839b54a4c9197dca8c9e2226e84, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=5bdf239fa2bc4f1b9e484f916c438c1a) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:46.216486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c89b7-508e-4e31-94fa-c1c6fd0ec24a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:46.218192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c89b7-508e-4e31-94fa-c1c6fd0ec24a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:46.224520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd54085-fd5d-4147-8a1d-3df93420bbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:46.224999 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 493/994] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:46 2022] GET /identity/v3/role_assignments?user.id=5bdf239fa2bc4f1b9e484f916c438c1a&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=44488839b54a4c9197dca8c9e2226e84 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:46.226207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd54085-fd5d-4147-8a1d-3df93420bbac None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:46.226496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd54085-fd5d-4147-8a1d-3df93420bbac None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:46.226773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd54085-fd5d-4147-8a1d-3df93420bbac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:46.227573 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd54085-fd5d-4147-8a1d-3df93420bbac None admin] RBAC: Adding query filter params (interface=admin, service_id=695b96afcb414be5900b96d7975b50a9, region_id=RegionOne) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:46.227903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd54085-fd5d-4147-8a1d-3df93420bbac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:46.229264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd54085-fd5d-4147-8a1d-3df93420bbac None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:46.241386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 502/995] 10.222.0.79 () {56 vars in 1249 bytes} [Mon Jul 4 09:56:46 2022] GET /identity/v3/endpoints?service_id=695b96afcb414be5900b96d7975b50a9&interface=admin®ion_id=RegionOne => generated 193 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:46.244871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c89b7-508e-4e31-94fa-c1c6fd0ec24a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:46.316340 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:46.361364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c89b7-508e-4e31-94fa-c1c6fd0ec24a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:46.400532 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195b8b8-53e4-4574-97c0-5c155a496565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:46.401579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195b8b8-53e4-4574-97c0-5c155a496565 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:46.401950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195b8b8-53e4-4574-97c0-5c155a496565 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:46.402227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195b8b8-53e4-4574-97c0-5c155a496565 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:46.402915 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195b8b8-53e4-4574-97c0-5c155a496565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:46.404305 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195b8b8-53e4-4574-97c0-5c155a496565 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:46.411627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 494/996] 10.222.0.79 () {56 vars in 1087 bytes} [Mon Jul 4 09:56:46 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:56:47.473227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83490e-7e99-41f0-a393-4c3ffeaf9bfe None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:47.473755 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83490e-7e99-41f0-a393-4c3ffeaf9bfe None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:47.473755 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83490e-7e99-41f0-a393-4c3ffeaf9bfe None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:47.475665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 503/997] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:47.481264 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e18417-0a4e-4c0a-becc-ce5493b122a3 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:47.481709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e18417-0a4e-4c0a-becc-ce5493b122a3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:47.481876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e18417-0a4e-4c0a-becc-ce5493b122a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:47.500315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c9e18417-0a4e-4c0a-becc-ce5493b122a3 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:47.501540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c9e18417-0a4e-4c0a-becc-ce5493b122a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:47.573651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 495/998] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:47.597101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83490e-7e99-41f0-a393-4c3ffeaf9bfe None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:47.700074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83490e-7e99-41f0-a393-4c3ffeaf9bfe None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:47.736192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4c422-26ab-40a6-aeca-467d05f6e86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:47.736998 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4c422-26ab-40a6-aeca-467d05f6e86b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:47.737279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4c422-26ab-40a6-aeca-467d05f6e86b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:47.737555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4c422-26ab-40a6-aeca-467d05f6e86b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:47.738248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4c422-26ab-40a6-aeca-467d05f6e86b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:47.739646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4c422-26ab-40a6-aeca-467d05f6e86b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-15a4c422-26ab-40a6-aeca-467d05f6e86b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:47.751371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:47.752334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:56:47.753206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:47.757472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 504/999] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:47.758777 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e18417-0a4e-4c0a-becc-ce5493b122a3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:47.816385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:47.862360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e18417-0a4e-4c0a-becc-ce5493b122a3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:47.896551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd7178-7daa-4842-83ad-73d5948fba38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:47.897206 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd7178-7daa-4842-83ad-73d5948fba38 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:47.897474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd7178-7daa-4842-83ad-73d5948fba38 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:47.897786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd7178-7daa-4842-83ad-73d5948fba38 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:47.898484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd7178-7daa-4842-83ad-73d5948fba38 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:47.898784 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd7178-7daa-4842-83ad-73d5948fba38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:47.900168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd7178-7daa-4842-83ad-73d5948fba38 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:47.905416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 496/1000] 10.222.0.79 () {56 vars in 1112 bytes} [Mon Jul 4 09:56:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:47.911131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4c422-26ab-40a6-aeca-467d05f6e86b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:47.958334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:48.014438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4c422-26ab-40a6-aeca-467d05f6e86b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:48.047982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6c4d8-efc7-4918-8913-1bb5dffd08f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:48.048652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6c4d8-efc7-4918-8913-1bb5dffd08f3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:48.048927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6c4d8-efc7-4918-8913-1bb5dffd08f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:48.049198 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6c4d8-efc7-4918-8913-1bb5dffd08f3 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:48.049929 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6c4d8-efc7-4918-8913-1bb5dffd08f3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:48.050231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6c4d8-efc7-4918-8913-1bb5dffd08f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:48.051594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6c4d8-efc7-4918-8913-1bb5dffd08f3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:48.056775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 505/1001] 10.222.0.79 () {56 vars in 1112 bytes} [Mon Jul 4 09:56:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:48.062602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd7178-7daa-4842-83ad-73d5948fba38 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:48.176965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd7178-7daa-4842-83ad-73d5948fba38 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:48.197882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:48.214090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925d248-909d-4a6b-9782-a939deb5564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:48.214766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925d248-909d-4a6b-9782-a939deb5564a None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:48.215049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925d248-909d-4a6b-9782-a939deb5564a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:48.215328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925d248-909d-4a6b-9782-a939deb5564a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:48.216065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925d248-909d-4a6b-9782-a939deb5564a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:48.217626 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925d248-909d-4a6b-9782-a939deb5564a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:48.245652 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 497/1002] 10.222.0.79 () {60 vars in 1144 bytes} [Mon Jul 4 09:56:48 2022] POST /identity/v3/endpoints => generated 334 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:56:48.296185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f346d2-6dd9-4c30-8ad7-d5b2e19602d8 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:48.296593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f346d2-6dd9-4c30-8ad7-d5b2e19602d8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:48.296711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f346d2-6dd9-4c30-8ad7-d5b2e19602d8 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:48.298240 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 506/1003] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:48.304379 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bb4d5-347e-4db3-b671-b770daa20bc5 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:48.304705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bb4d5-347e-4db3-b671-b770daa20bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:48.304911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bb4d5-347e-4db3-b671-b770daa20bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:48.323959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-232bb4d5-347e-4db3-b671-b770daa20bc5 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:48.325187 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-232bb4d5-347e-4db3-b671-b770daa20bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:48.411721 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 498/1004] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:48 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:48.426957 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172deef-10f6-46f1-bc73-b29791fbaf05 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:48.427252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172deef-10f6-46f1-bc73-b29791fbaf05 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:48.427537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172deef-10f6-46f1-bc73-b29791fbaf05 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:48.428965 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 507/1005] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:48.434401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ff57c-1490-4ac9-b498-605d5808d5d8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:48.434624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172deef-10f6-46f1-bc73-b29791fbaf05 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:48.434724 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ff57c-1490-4ac9-b498-605d5808d5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:48.434974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ff57c-1490-4ac9-b498-605d5808d5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:48.452969 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-579ff57c-1490-4ac9-b498-605d5808d5d8 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:48.454205 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-579ff57c-1490-4ac9-b498-605d5808d5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:48.533374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 499/1006] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:48 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:48.553932 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172deef-10f6-46f1-bc73-b29791fbaf05 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:48.557185 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ff57c-1490-4ac9-b498-605d5808d5d8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:48.564510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:48.588895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:48.592310 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d984-b042-437c-afed-60a1388c4452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:48.593808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d984-b042-437c-afed-60a1388c4452 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:48.594237 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d984-b042-437c-afed-60a1388c4452 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:48.594365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d984-b042-437c-afed-60a1388c4452 None admin] PATH_INFO: `/v3/services/image` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:48.595080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12d984-b042-437c-afed-60a1388c4452 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:48.596626 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12d984-b042-437c-afed-60a1388c4452 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-5f12d984-b042-437c-afed-60a1388c4452 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:48.609220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:48.610288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:48.611632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 508/1007] 10.222.0.79 () {56 vars in 1105 bytes} [Mon Jul 4 09:56:48 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:48.616338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d984-b042-437c-afed-60a1388c4452 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:48.665323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ff57c-1490-4ac9-b498-605d5808d5d8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:48.700963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946c76e-55be-4c88-a521-aaab0c41de84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:48.701813 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946c76e-55be-4c88-a521-aaab0c41de84 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:48.702253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946c76e-55be-4c88-a521-aaab0c41de84 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:48.702692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946c76e-55be-4c88-a521-aaab0c41de84 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:48.703354 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946c76e-55be-4c88-a521-aaab0c41de84 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:48.704749 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946c76e-55be-4c88-a521-aaab0c41de84 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b946c76e-55be-4c88-a521-aaab0c41de84 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:48.717108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:48.718239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:48.719097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 500/1008] 10.222.0.79 () {56 vars in 1126 bytes} [Mon Jul 4 09:56:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:48.724319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946c76e-55be-4c88-a521-aaab0c41de84 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:48.730848 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d984-b042-437c-afed-60a1388c4452 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:48.765518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dca8f-8fab-46d7-b044-9367fa9c7845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:48.766214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dca8f-8fab-46d7-b044-9367fa9c7845 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:48.766496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dca8f-8fab-46d7-b044-9367fa9c7845 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:48.766779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dca8f-8fab-46d7-b044-9367fa9c7845 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:48.767486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dca8f-8fab-46d7-b044-9367fa9c7845 None admin] RBAC: Adding query filter params (name=image) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:48.767795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dca8f-8fab-46d7-b044-9367fa9c7845 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:48.769166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dca8f-8fab-46d7-b044-9367fa9c7845 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:48.774092 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 509/1009] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:48.779865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dca8f-8fab-46d7-b044-9367fa9c7845 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:48.820868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:48.828846 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:48.835107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946c76e-55be-4c88-a521-aaab0c41de84 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:48.871148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a696d91-c771-45b7-a274-300d03bd24fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:48.871809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a696d91-c771-45b7-a274-300d03bd24fc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:48.872093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a696d91-c771-45b7-a274-300d03bd24fc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:48.872372 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a696d91-c771-45b7-a274-300d03bd24fc None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:48.873523 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a696d91-c771-45b7-a274-300d03bd24fc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:48.874010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a696d91-c771-45b7-a274-300d03bd24fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:48.875590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a696d91-c771-45b7-a274-300d03bd24fc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:48.881116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 501/1010] 10.222.0.79 () {56 vars in 1122 bytes} [Mon Jul 4 09:56:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:48.885609 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dca8f-8fab-46d7-b044-9367fa9c7845 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:48.886456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a696d91-c771-45b7-a274-300d03bd24fc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:48.920596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6e633-f5ed-41e5-859a-e5c1074ae376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:48.921273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6e633-f5ed-41e5-859a-e5c1074ae376 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:48.921559 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6e633-f5ed-41e5-859a-e5c1074ae376 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:48.921869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6e633-f5ed-41e5-859a-e5c1074ae376 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:48.922576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6e633-f5ed-41e5-859a-e5c1074ae376 None admin] RBAC: Adding query filter params (type=image) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:48.922885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6e633-f5ed-41e5-859a-e5c1074ae376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:48.924259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6e633-f5ed-41e5-859a-e5c1074ae376 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:48.929315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 510/1011] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:48.995166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a696d91-c771-45b7-a274-300d03bd24fc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:49.033461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a3723-968d-4205-9255-1eab751f6324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:49.034183 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a3723-968d-4205-9255-1eab751f6324 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:49.034468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a3723-968d-4205-9255-1eab751f6324 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:49.034745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a3723-968d-4205-9255-1eab751f6324 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:49.035460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a3723-968d-4205-9255-1eab751f6324 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:49.035767 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a3723-968d-4205-9255-1eab751f6324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:49.037166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a3723-968d-4205-9255-1eab751f6324 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:49.047758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 502/1012] 10.222.0.79 () {56 vars in 1122 bytes} [Mon Jul 4 09:56:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:49.049377 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6e633-f5ed-41e5-859a-e5c1074ae376 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:49.164701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6e633-f5ed-41e5-859a-e5c1074ae376 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:49.166254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:49.200181 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae323bb8-4b23-4baa-8194-f201fe2df4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:49.200838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae323bb8-4b23-4baa-8194-f201fe2df4aa None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:49.201123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae323bb8-4b23-4baa-8194-f201fe2df4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:49.201400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae323bb8-4b23-4baa-8194-f201fe2df4aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:49.202439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae323bb8-4b23-4baa-8194-f201fe2df4aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:49.203820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae323bb8-4b23-4baa-8194-f201fe2df4aa None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:49.219642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 511/1013] 10.222.0.79 () {60 vars in 1144 bytes} [Mon Jul 4 09:56:49 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:56:50.350906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b87919-3ed4-4c83-9f64-4f40255210d8 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:50.353860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b87919-3ed4-4c83-9f64-4f40255210d8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:50.354142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b87919-3ed4-4c83-9f64-4f40255210d8 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:50.355566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 503/1014] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:50.361399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bba15d-b38d-48ca-906e-a9b2fa8c199c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:50.362343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bba15d-b38d-48ca-906e-a9b2fa8c199c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:50.362623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bba15d-b38d-48ca-906e-a9b2fa8c199c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:50.380350 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a9bba15d-b38d-48ca-906e-a9b2fa8c199c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:50.381522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a9bba15d-b38d-48ca-906e-a9b2fa8c199c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:50.457821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 512/1015] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:50 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:50.477652 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b87919-3ed4-4c83-9f64-4f40255210d8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:50.504380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:50.582456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b87919-3ed4-4c83-9f64-4f40255210d8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:50.616033 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ad00c-714d-4db3-8b42-edc0bb278a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:50.616832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ad00c-714d-4db3-8b42-edc0bb278a9d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:50.617125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ad00c-714d-4db3-8b42-edc0bb278a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:50.617393 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ad00c-714d-4db3-8b42-edc0bb278a9d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:50.631215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ad00c-714d-4db3-8b42-edc0bb278a9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d48ad00c-714d-4db3-8b42-edc0bb278a9d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:50.640043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:50.641034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:50.641910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:50.641910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:50.641910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:50.641910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:50.641910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:50.641910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:50.641910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:50.641910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:50.641910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:50.641910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:50.641910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:56:50.641910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:50.641910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 504/1016] 10.222.0.79 () {56 vars in 1120 bytes} [Mon Jul 4 09:56:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:50.648923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bba15d-b38d-48ca-906e-a9b2fa8c199c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:50.740273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:50.754856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bba15d-b38d-48ca-906e-a9b2fa8c199c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:50.790059 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dead67-bb60-44fd-a746-b7988c138013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:50.790739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dead67-bb60-44fd-a746-b7988c138013 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:50.791025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dead67-bb60-44fd-a746-b7988c138013 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:50.791302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dead67-bb60-44fd-a746-b7988c138013 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:50.792057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dead67-bb60-44fd-a746-b7988c138013 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:50.792364 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dead67-bb60-44fd-a746-b7988c138013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:50.793768 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dead67-bb60-44fd-a746-b7988c138013 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:50.807057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 513/1017] 10.222.0.79 () {56 vars in 1115 bytes} [Mon Jul 4 09:56:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:50.808062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ad00c-714d-4db3-8b42-edc0bb278a9d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:50.894455 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:50.911349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ad00c-714d-4db3-8b42-edc0bb278a9d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:50.945400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a2d90-da6b-4daf-99b9-80faf19f6e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:50.946232 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a2d90-da6b-4daf-99b9-80faf19f6e1a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:50.946520 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a2d90-da6b-4daf-99b9-80faf19f6e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:50.946797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a2d90-da6b-4daf-99b9-80faf19f6e1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:50.957275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a2d90-da6b-4daf-99b9-80faf19f6e1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:50.958774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a2d90-da6b-4daf-99b9-80faf19f6e1a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:50.973076 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe4e55-2b73-4dcd-bb75-e10f071a2687 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:50.973375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe4e55-2b73-4dcd-bb75-e10f071a2687 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:50.973676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe4e55-2b73-4dcd-bb75-e10f071a2687 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:50.975528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 514/1018] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-cc5a2d90-da6b-4daf-99b9-80faf19f6e1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:50.976557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:50.977403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:50.983163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:50.983163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:50.983163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:50.983163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:50.983163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 505/1019] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:50.983454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab91a86-cefd-4e7b-82ed-94de7b21bdb8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:50.983454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab91a86-cefd-4e7b-82ed-94de7b21bdb8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:50.983454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab91a86-cefd-4e7b-82ed-94de7b21bdb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:50.983642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a2d90-da6b-4daf-99b9-80faf19f6e1a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:50.999625 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-bab91a86-cefd-4e7b-82ed-94de7b21bdb8 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:51.000750 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bab91a86-cefd-4e7b-82ed-94de7b21bdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:51.067170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 515/1020] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:50 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:51.091093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab91a86-cefd-4e7b-82ed-94de7b21bdb8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:51.093453 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a2d90-da6b-4daf-99b9-80faf19f6e1a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:51.133938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f8bb5-81d5-4f34-bc8a-6b5edcb362f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:51.135881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f8bb5-81d5-4f34-bc8a-6b5edcb362f5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:51.136167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f8bb5-81d5-4f34-bc8a-6b5edcb362f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:51.136445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f8bb5-81d5-4f34-bc8a-6b5edcb362f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:51.137170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f8bb5-81d5-4f34-bc8a-6b5edcb362f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:51.137474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f8bb5-81d5-4f34-bc8a-6b5edcb362f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:51.138887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f8bb5-81d5-4f34-bc8a-6b5edcb362f5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:51.142019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:51.153100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 506/1021] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:51.158381 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f8bb5-81d5-4f34-bc8a-6b5edcb362f5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:51.186356 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:51.208939 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab91a86-cefd-4e7b-82ed-94de7b21bdb8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:51.244524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f61e10-0190-4284-8df0-39880ed2d301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:51.245200 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f61e10-0190-4284-8df0-39880ed2d301 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:51.245487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f61e10-0190-4284-8df0-39880ed2d301 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:51.245813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f61e10-0190-4284-8df0-39880ed2d301 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:51.246544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f61e10-0190-4284-8df0-39880ed2d301 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:51.247933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f61e10-0190-4284-8df0-39880ed2d301 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:51.269765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 516/1022] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:56:51 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:56:51.269765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3176f3-c06d-4f1f-b38f-042712238778 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:51.269765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3176f3-c06d-4f1f-b38f-042712238778 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:51.269765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3176f3-c06d-4f1f-b38f-042712238778 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:51.269765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 517/1023] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:51.273493 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c84ac-566f-4cec-a842-094b9f077d2f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:51.273859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c84ac-566f-4cec-a842-094b9f077d2f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:51.273961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f8bb5-81d5-4f34-bc8a-6b5edcb362f5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:51.274126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c84ac-566f-4cec-a842-094b9f077d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:51.293083 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f00c84ac-566f-4cec-a842-094b9f077d2f None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:51.294342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f00c84ac-566f-4cec-a842-094b9f077d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:51.324815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fa657-e850-4609-893e-866f83b80e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:51.325489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fa657-e850-4609-893e-866f83b80e7e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:51.325794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fa657-e850-4609-893e-866f83b80e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:51.326118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fa657-e850-4609-893e-866f83b80e7e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:51.335000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fa657-e850-4609-893e-866f83b80e7e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:51.336496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fa657-e850-4609-893e-866f83b80e7e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-183fa657-e850-4609-893e-866f83b80e7e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:51.353004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:51.354768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:51.355938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:51.355938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:51.355938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:51.355938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:51.355938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:51.355938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:51.355938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:51.355938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:51.355938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:51.355938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:51.355938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 507/1024] 10.222.0.79 () {56 vars in 1093 bytes} [Mon Jul 4 09:56:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:51.360287 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fa657-e850-4609-893e-866f83b80e7e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:51.379017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 518/1025] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:51.404748 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c84ac-566f-4cec-a842-094b9f077d2f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:51.419067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:51.477386 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fa657-e850-4609-893e-866f83b80e7e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:51.483184 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:51.517326 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1e173-4802-4c31-a9ed-0c0e204dd7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:51.518187 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1e173-4802-4c31-a9ed-0c0e204dd7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:51.518467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1e173-4802-4c31-a9ed-0c0e204dd7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:51.518762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1e173-4802-4c31-a9ed-0c0e204dd7d3 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:51.519566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1e173-4802-4c31-a9ed-0c0e204dd7d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:51.519923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1e173-4802-4c31-a9ed-0c0e204dd7d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:51.520856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c84ac-566f-4cec-a842-094b9f077d2f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:51.521322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1e173-4802-4c31-a9ed-0c0e204dd7d3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:51.547294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 508/1026] 10.222.0.79 () {56 vars in 1133 bytes} [Mon Jul 4 09:56:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:51.549004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1e173-4802-4c31-a9ed-0c0e204dd7d3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:51.555543 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48507eb2-a15c-42d7-98c7-a03d8b8b5ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:51.556351 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48507eb2-a15c-42d7-98c7-a03d8b8b5ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:51.556632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48507eb2-a15c-42d7-98c7-a03d8b8b5ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:51.556906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48507eb2-a15c-42d7-98c7-a03d8b8b5ca3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:51.558118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48507eb2-a15c-42d7-98c7-a03d8b8b5ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:51.559551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48507eb2-a15c-42d7-98c7-a03d8b8b5ca3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:51.565862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 519/1027] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:51.566866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48507eb2-a15c-42d7-98c7-a03d8b8b5ca3 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:51.653212 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1e173-4802-4c31-a9ed-0c0e204dd7d3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:51.664628 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:51.674773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48507eb2-a15c-42d7-98c7-a03d8b8b5ca3 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:51.692563 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d0a3d-6600-4d82-82ce-859d7cf737a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:51.693226 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d0a3d-6600-4d82-82ce-859d7cf737a5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:51.693510 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d0a3d-6600-4d82-82ce-859d7cf737a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:51.693815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d0a3d-6600-4d82-82ce-859d7cf737a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:51.712167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0cdb6-ae1e-4f0f-9ca3-e03e13f9c921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:51.712833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0cdb6-ae1e-4f0f-9ca3-e03e13f9c921 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:51.713116 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0cdb6-ae1e-4f0f-9ca3-e03e13f9c921 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:51.713418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0cdb6-ae1e-4f0f-9ca3-e03e13f9c921 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:51.714122 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0cdb6-ae1e-4f0f-9ca3-e03e13f9c921 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:51.715579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0cdb6-ae1e-4f0f-9ca3-e03e13f9c921 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:51.723825 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:51.749667 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 520/1028] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:56:51 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:56:51.781201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d0a3d-6600-4d82-82ce-859d7cf737a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:51.782895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d0a3d-6600-4d82-82ce-859d7cf737a5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:51.899998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 509/1029] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:51 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 09:56:51.901770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0cdb6-ae1e-4f0f-9ca3-e03e13f9c921 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:52.010270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0cdb6-ae1e-4f0f-9ca3-e03e13f9c921 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:52.046477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc223e-deb4-4d78-8a42-aaed44d6c231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:52.047202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc223e-deb4-4d78-8a42-aaed44d6c231 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:52.047486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc223e-deb4-4d78-8a42-aaed44d6c231 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:52.047759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc223e-deb4-4d78-8a42-aaed44d6c231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:52.059540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc223e-deb4-4d78-8a42-aaed44d6c231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:52.061087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc223e-deb4-4d78-8a42-aaed44d6c231 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-8cfc223e-deb4-4d78-8a42-aaed44d6c231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:52.078137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:52.079217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:52.080100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:52.080100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:52.080100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:52.080100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:52.081738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 521/1030] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:52.085354 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d0a3d-6600-4d82-82ce-859d7cf737a5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:52.098017 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:52.193404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d0a3d-6600-4d82-82ce-859d7cf737a5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:52.229869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4eb121-1090-4a49-a5fa-74f85ac8e74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:52.230590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eb121-1090-4a49-a5fa-74f85ac8e74d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:52.230873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eb121-1090-4a49-a5fa-74f85ac8e74d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:52.231144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eb121-1090-4a49-a5fa-74f85ac8e74d None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:52.231867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4eb121-1090-4a49-a5fa-74f85ac8e74d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:52.232252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4eb121-1090-4a49-a5fa-74f85ac8e74d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:52.233978 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4eb121-1090-4a49-a5fa-74f85ac8e74d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:52.245777 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 510/1031] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:52.251385 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc223e-deb4-4d78-8a42-aaed44d6c231 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:52.297396 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:52.359672 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc223e-deb4-4d78-8a42-aaed44d6c231 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:52.394565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0a08f-0df8-4d81-9ce2-905d7e766777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:52.395394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0a08f-0df8-4d81-9ce2-905d7e766777 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:52.395662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0a08f-0df8-4d81-9ce2-905d7e766777 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:52.395933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0a08f-0df8-4d81-9ce2-905d7e766777 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:52.403847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0a08f-0df8-4d81-9ce2-905d7e766777 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:52.405404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0a08f-0df8-4d81-9ce2-905d7e766777 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b7a0a08f-0df8-4d81-9ce2-905d7e766777 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:52.420642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:52.421906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:52.422839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:52.422839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:52.422839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:52.422839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:52.422839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:52.422839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:52.422839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:52.422839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:52.422839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 522/1032] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:52.427805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4eb121-1090-4a49-a5fa-74f85ac8e74d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:52.500730 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:52.533941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4eb121-1090-4a49-a5fa-74f85ac8e74d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:52.568625 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d7b6e-0baa-4c6e-8257-1c58ecd0fe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:52.569508 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d7b6e-0baa-4c6e-8257-1c58ecd0fe12 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:52.569831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d7b6e-0baa-4c6e-8257-1c58ecd0fe12 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:52.570114 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d7b6e-0baa-4c6e-8257-1c58ecd0fe12 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:52.570896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d7b6e-0baa-4c6e-8257-1c58ecd0fe12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:52.571210 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d7b6e-0baa-4c6e-8257-1c58ecd0fe12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:52.572648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d7b6e-0baa-4c6e-8257-1c58ecd0fe12 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:52.585094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 511/1033] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:52.590989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0a08f-0df8-4d81-9ce2-905d7e766777 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:52.693950 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0a08f-0df8-4d81-9ce2-905d7e766777 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:52.695461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:52.731504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16efa8-af6e-4610-ada0-6f1e95607e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:52.732166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16efa8-af6e-4610-ada0-6f1e95607e2d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:52.732847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16efa8-af6e-4610-ada0-6f1e95607e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:52.733129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16efa8-af6e-4610-ada0-6f1e95607e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:52.734058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16efa8-af6e-4610-ada0-6f1e95607e2d None admin] RBAC: Adding query filter params (role.id=44488839b54a4c9197dca8c9e2226e84, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=ffc2409cb7fc4373a74ddfb48d501d30) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:52.734376 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16efa8-af6e-4610-ada0-6f1e95607e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:52.735818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16efa8-af6e-4610-ada0-6f1e95607e2d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:52.749647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 523/1034] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:52 2022] GET /identity/v3/role_assignments?user.id=ffc2409cb7fc4373a74ddfb48d501d30&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=44488839b54a4c9197dca8c9e2226e84 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:53.317885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d928b93-2729-466d-90c7-8e027d453d22 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:53.318230 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d928b93-2729-466d-90c7-8e027d453d22 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:53.318440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d928b93-2729-466d-90c7-8e027d453d22 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:53.319846 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 512/1035] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:53.325221 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c7b6-ba50-40a2-9b27-b42f4a402d64 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:53.325511 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c7b6-ba50-40a2-9b27-b42f4a402d64 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:53.325806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c7b6-ba50-40a2-9b27-b42f4a402d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:53.344141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-46f8c7b6-ba50-40a2-9b27-b42f4a402d64 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:53.345342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-46f8c7b6-ba50-40a2-9b27-b42f4a402d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:53.418618 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 524/1036] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:53.441590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d928b93-2729-466d-90c7-8e027d453d22 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:53.544400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d928b93-2729-466d-90c7-8e027d453d22 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:53.573123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:53.580587 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa60337-8b91-480e-a542-42cb7d173306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:53.581386 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa60337-8b91-480e-a542-42cb7d173306 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:53.581684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa60337-8b91-480e-a542-42cb7d173306 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:53.581980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa60337-8b91-480e-a542-42cb7d173306 None admin] PATH_INFO: `/v3/services/image` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:53.582628 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa60337-8b91-480e-a542-42cb7d173306 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:53.584021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa60337-8b91-480e-a542-42cb7d173306 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1aa60337-8b91-480e-a542-42cb7d173306 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:53.595694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:53.596701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:56:53.597541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:53.601995 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 513/1037] 10.222.0.79 () {56 vars in 1105 bytes} [Mon Jul 4 09:56:53 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:53.603294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8c7b6-ba50-40a2-9b27-b42f4a402d64 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:53.698590 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:53.705848 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8c7b6-ba50-40a2-9b27-b42f4a402d64 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:53.741044 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a627b3-a40d-41ce-9747-d3fee5efb852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:53.741736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a627b3-a40d-41ce-9747-d3fee5efb852 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:53.742025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a627b3-a40d-41ce-9747-d3fee5efb852 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:53.742300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a627b3-a40d-41ce-9747-d3fee5efb852 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:53.743011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a627b3-a40d-41ce-9747-d3fee5efb852 None admin] RBAC: Adding query filter params (name=image) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:53.743318 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a627b3-a40d-41ce-9747-d3fee5efb852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:53.744699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a627b3-a40d-41ce-9747-d3fee5efb852 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:53.750095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 525/1038] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:53.755694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa60337-8b91-480e-a542-42cb7d173306 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:53.852951 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c63efa-7ad7-43ce-8252-39c0735403cc None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:53.853308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c63efa-7ad7-43ce-8252-39c0735403cc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:53.853537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c63efa-7ad7-43ce-8252-39c0735403cc None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:53.855213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 526/1039] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:53.860094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5fbb1-3d5e-420c-adf6-095484fc873e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:53.860392 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5fbb1-3d5e-420c-adf6-095484fc873e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:53.860679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5fbb1-3d5e-420c-adf6-095484fc873e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:53.870014 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa60337-8b91-480e-a542-42cb7d173306 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:53.877527 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-24e5fbb1-3d5e-420c-adf6-095484fc873e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:53.878755 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-24e5fbb1-3d5e-420c-adf6-095484fc873e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:53.905509 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64c435-491c-4efa-9a61-4d447f50043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:53.906459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64c435-491c-4efa-9a61-4d447f50043e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:53.906882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64c435-491c-4efa-9a61-4d447f50043e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:53.907295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64c435-491c-4efa-9a61-4d447f50043e None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:53.908136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64c435-491c-4efa-9a61-4d447f50043e None admin] RBAC: Adding query filter params (type=image) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:53.908594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64c435-491c-4efa-9a61-4d447f50043e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:53.910152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64c435-491c-4efa-9a61-4d447f50043e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:53.915176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 514/1040] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:53.921127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64c435-491c-4efa-9a61-4d447f50043e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:53.953687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 527/1041] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:53.979171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5fbb1-3d5e-420c-adf6-095484fc873e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:53.981012 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:54.026285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64c435-491c-4efa-9a61-4d447f50043e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:54.062502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961a731-3ffd-40f6-89f4-f5383a4f6367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:54.063277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961a731-3ffd-40f6-89f4-f5383a4f6367 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:54.063562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961a731-3ffd-40f6-89f4-f5383a4f6367 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:54.063959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961a731-3ffd-40f6-89f4-f5383a4f6367 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:54.064753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961a731-3ffd-40f6-89f4-f5383a4f6367 None admin] RBAC: Adding query filter params (interface=public, service_id=3bb5b117044e4a5290269e9d3c51d328, region_id=RegionOne) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:54.065069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961a731-3ffd-40f6-89f4-f5383a4f6367 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:54.066600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961a731-3ffd-40f6-89f4-f5383a4f6367 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:54.072480 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:54.078485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 515/1042] 10.222.0.79 () {56 vars in 1251 bytes} [Mon Jul 4 09:56:53 2022] GET /identity/v3/endpoints?service_id=3bb5b117044e4a5290269e9d3c51d328&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:54.079451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961a731-3ffd-40f6-89f4-f5383a4f6367 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:54.087836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5fbb1-3d5e-420c-adf6-095484fc873e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:54.124520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2145fc3-80a5-461b-8b40-8e7b528f9f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:54.125325 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2145fc3-80a5-461b-8b40-8e7b528f9f84 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:54.125607 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2145fc3-80a5-461b-8b40-8e7b528f9f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:54.125923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2145fc3-80a5-461b-8b40-8e7b528f9f84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:54.127114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2145fc3-80a5-461b-8b40-8e7b528f9f84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:54.128534 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2145fc3-80a5-461b-8b40-8e7b528f9f84 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:54.132770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 528/1043] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:54.135921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2145fc3-80a5-461b-8b40-8e7b528f9f84 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:54.197050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961a731-3ffd-40f6-89f4-f5383a4f6367 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:54.221687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:54.234409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb39107-1baf-4c25-b560-2085db73f345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:54.235085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb39107-1baf-4c25-b560-2085db73f345 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:54.235367 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb39107-1baf-4c25-b560-2085db73f345 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:54.235643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb39107-1baf-4c25-b560-2085db73f345 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:54.236329 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb39107-1baf-4c25-b560-2085db73f345 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:54.237821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb39107-1baf-4c25-b560-2085db73f345 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:54.239926 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2145fc3-80a5-461b-8b40-8e7b528f9f84 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:54.244622 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 516/1044] 10.222.0.79 () {56 vars in 1087 bytes} [Mon Jul 4 09:56:54 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:56:54.277567 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82ece3-13c1-4ad2-a2ec-5f17f5e11084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:54.278305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82ece3-13c1-4ad2-a2ec-5f17f5e11084 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:54.278591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82ece3-13c1-4ad2-a2ec-5f17f5e11084 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:54.278870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82ece3-13c1-4ad2-a2ec-5f17f5e11084 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:54.279692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82ece3-13c1-4ad2-a2ec-5f17f5e11084 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:54.281368 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82ece3-13c1-4ad2-a2ec-5f17f5e11084 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:54.304337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 529/1045] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:56:54 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:56:54.767825 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a2008-c85f-4fd5-ba4e-0f570b5b6fd4 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:54.768407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a2008-c85f-4fd5-ba4e-0f570b5b6fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:54.768407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a2008-c85f-4fd5-ba4e-0f570b5b6fd4 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:54.769945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 517/1046] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:54.775262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bdf8f-f069-4be6-b8ea-b0eea9f2d8e3 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:54.775686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bdf8f-f069-4be6-b8ea-b0eea9f2d8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:54.775877 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bdf8f-f069-4be6-b8ea-b0eea9f2d8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:54.793056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1a3bdf8f-f069-4be6-b8ea-b0eea9f2d8e3 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:54.794275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1a3bdf8f-f069-4be6-b8ea-b0eea9f2d8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:54.804673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:54.869729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 530/1047] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:54.889854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a2008-c85f-4fd5-ba4e-0f570b5b6fd4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:54.995353 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a2008-c85f-4fd5-ba4e-0f570b5b6fd4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:55.029398 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2018ce0-9c82-4114-a4fb-9d21d1aa1c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:55.030343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2018ce0-9c82-4114-a4fb-9d21d1aa1c95 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:55.030627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2018ce0-9c82-4114-a4fb-9d21d1aa1c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:55.030899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2018ce0-9c82-4114-a4fb-9d21d1aa1c95 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:55.042033 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2018ce0-9c82-4114-a4fb-9d21d1aa1c95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-c2018ce0-9c82-4114-a4fb-9d21d1aa1c95 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:55.050111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:55.051028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:55.051918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:55.051918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:55.051918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:55.051918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:55.051918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:55.051918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:55.051918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:55.051918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:55.051918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:55.051918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:55.051918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:56:55.051918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:55.053732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 518/1048] 10.222.0.79 () {56 vars in 1120 bytes} [Mon Jul 4 09:56:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:55.058035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bdf8f-f069-4be6-b8ea-b0eea9f2d8e3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:55.169358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bdf8f-f069-4be6-b8ea-b0eea9f2d8e3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:55.204110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd11919-7018-477f-abf9-47c8a1c56fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:55.204773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd11919-7018-477f-abf9-47c8a1c56fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:55.205059 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd11919-7018-477f-abf9-47c8a1c56fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:55.205326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd11919-7018-477f-abf9-47c8a1c56fb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:55.206108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd11919-7018-477f-abf9-47c8a1c56fb7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:55.206420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd11919-7018-477f-abf9-47c8a1c56fb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:55.207791 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd11919-7018-477f-abf9-47c8a1c56fb7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:55.216278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 531/1049] 10.222.0.79 () {56 vars in 1115 bytes} [Mon Jul 4 09:56:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:55.222384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2018ce0-9c82-4114-a4fb-9d21d1aa1c95 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:55.272285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:55.327779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2018ce0-9c82-4114-a4fb-9d21d1aa1c95 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:55.363570 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3ca08-2099-44a2-85af-138407caedee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:55.364380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3ca08-2099-44a2-85af-138407caedee None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:55.364664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3ca08-2099-44a2-85af-138407caedee None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:55.364939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3ca08-2099-44a2-85af-138407caedee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:55.375792 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3ca08-2099-44a2-85af-138407caedee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:55.377384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3ca08-2099-44a2-85af-138407caedee None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-f3b3ca08-2099-44a2-85af-138407caedee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:55.396548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:55.397541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:55.398492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:55.398492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:55.398492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:55.398492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:55.398492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 519/1050] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:55.405252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd11919-7018-477f-abf9-47c8a1c56fb7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:55.415591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:55.518063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd11919-7018-477f-abf9-47c8a1c56fb7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:55.552792 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd775e05-6645-452d-89db-9d003847d629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:55.553498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd775e05-6645-452d-89db-9d003847d629 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:55.553808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd775e05-6645-452d-89db-9d003847d629 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:55.554087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd775e05-6645-452d-89db-9d003847d629 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:55.554810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd775e05-6645-452d-89db-9d003847d629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:55.555123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd775e05-6645-452d-89db-9d003847d629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:55.556503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd775e05-6645-452d-89db-9d003847d629 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:55.568574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 532/1051] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:55.574217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3ca08-2099-44a2-85af-138407caedee None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:55.670163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:55.674682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3ca08-2099-44a2-85af-138407caedee None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:55.708973 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad965a-2173-411d-b745-4d82387bfcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:55.709790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad965a-2173-411d-b745-4d82387bfcbd None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:55.710071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad965a-2173-411d-b745-4d82387bfcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:55.710344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad965a-2173-411d-b745-4d82387bfcbd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:55.717468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad965a-2173-411d-b745-4d82387bfcbd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:55.718963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad965a-2173-411d-b745-4d82387bfcbd None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-01ad965a-2173-411d-b745-4d82387bfcbd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:55.733576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:55.734643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:55.735590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:55.735590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:55.735590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:55.735590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:55.735590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:55.735590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:55.735590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:55.735590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:55.735590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:55.735590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:55.736867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 520/1052] 10.222.0.79 () {56 vars in 1093 bytes} [Mon Jul 4 09:56:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:55.740683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd775e05-6645-452d-89db-9d003847d629 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:55.811436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:55.855178 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd775e05-6645-452d-89db-9d003847d629 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:55.891798 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff0279-d086-4281-afb0-9b972297baeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:55.892476 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff0279-d086-4281-afb0-9b972297baeb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:55.892766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff0279-d086-4281-afb0-9b972297baeb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:55.893031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff0279-d086-4281-afb0-9b972297baeb None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:55.893926 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff0279-d086-4281-afb0-9b972297baeb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:55.894231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff0279-d086-4281-afb0-9b972297baeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:55.895659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff0279-d086-4281-afb0-9b972297baeb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:55.913354 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 533/1053] 10.222.0.79 () {56 vars in 1133 bytes} [Mon Jul 4 09:56:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:55.918922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad965a-2173-411d-b745-4d82387bfcbd None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:56.021689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad965a-2173-411d-b745-4d82387bfcbd None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:56.055912 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368499a-9d8d-4458-8c03-2a4f56630911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:56.056710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368499a-9d8d-4458-8c03-2a4f56630911 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:56.056990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368499a-9d8d-4458-8c03-2a4f56630911 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:56.057264 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368499a-9d8d-4458-8c03-2a4f56630911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:56.068012 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368499a-9d8d-4458-8c03-2a4f56630911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:56.069482 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368499a-9d8d-4458-8c03-2a4f56630911 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-f368499a-9d8d-4458-8c03-2a4f56630911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:56.087934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:56.088870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:56.089797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:56.089797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:56.089797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:56.089797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:56.094292 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 521/1054] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:56.095551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff0279-d086-4281-afb0-9b972297baeb None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:56.208335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff0279-d086-4281-afb0-9b972297baeb None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:56.232551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:56.245255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd205ce-401f-40ac-994e-f7be306979be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:56.245955 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd205ce-401f-40ac-994e-f7be306979be None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:56.246239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd205ce-401f-40ac-994e-f7be306979be None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:56.246512 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd205ce-401f-40ac-994e-f7be306979be None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:56.247230 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd205ce-401f-40ac-994e-f7be306979be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:56.247533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd205ce-401f-40ac-994e-f7be306979be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:56.249081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd205ce-401f-40ac-994e-f7be306979be None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:56.260676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 534/1055] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:56.266538 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368499a-9d8d-4458-8c03-2a4f56630911 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:56.279227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:56.324102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b2c27-dd07-4e95-b018-9d654141452b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:56.324397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b2c27-dd07-4e95-b018-9d654141452b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:56.324687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b2c27-dd07-4e95-b018-9d654141452b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:56.326430 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 535/1056] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:56:56.331478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d5012-46d5-442e-95bd-1cffa24abd96 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:56.331770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d5012-46d5-442e-95bd-1cffa24abd96 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:56.332052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d5012-46d5-442e-95bd-1cffa24abd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:56.350781 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-309d5012-46d5-442e-95bd-1cffa24abd96 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:56.352340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-309d5012-46d5-442e-95bd-1cffa24abd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:56.377995 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368499a-9d8d-4458-8c03-2a4f56630911 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:56.414697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3dbed-4b91-41f5-bc80-34b7a477dc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:56.415496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3dbed-4b91-41f5-bc80-34b7a477dc4e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:56.415829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3dbed-4b91-41f5-bc80-34b7a477dc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:56.416108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3dbed-4b91-41f5-bc80-34b7a477dc4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:56.421124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 536/1057] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:56.422494 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585201a4-ab2d-4a08-98aa-ee9cd377d94b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:56.422782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585201a4-ab2d-4a08-98aa-ee9cd377d94b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:56.423057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585201a4-ab2d-4a08-98aa-ee9cd377d94b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:56.423420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3dbed-4b91-41f5-bc80-34b7a477dc4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:56.424420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 537/1058] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:56.424954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3dbed-4b91-41f5-bc80-34b7a477dc4e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:56.430958 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ba562-75f9-43b5-a05f-d024eb7c62f3 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:56.431248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ba562-75f9-43b5-a05f-d024eb7c62f3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:56.431528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ba562-75f9-43b5-a05f-d024eb7c62f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-73c3dbed-4b91-41f5-bc80-34b7a477dc4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:56:56.444116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:56.445085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:56.446601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:56:56.446601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:56:56.446601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:56:56.446601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:56:56.446601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:56:56.446601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:56:56.446601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:56:56.446601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:56.446601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 522/1059] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:56:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:56.446601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3dbed-4b91-41f5-bc80-34b7a477dc4e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:56.448262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-785ba562-75f9-43b5-a05f-d024eb7c62f3 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:56.449442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-785ba562-75f9-43b5-a05f-d024eb7c62f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:56.514727 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:56.516279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 538/1060] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:56.518639 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ba562-75f9-43b5-a05f-d024eb7c62f3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:56.548451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3dbed-4b91-41f5-bc80-34b7a477dc4e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:56.563162 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:56.584095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73420107-5391-402e-a2cd-79ba54be9383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:56.584887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73420107-5391-402e-a2cd-79ba54be9383 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:56.585173 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73420107-5391-402e-a2cd-79ba54be9383 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:56.585450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73420107-5391-402e-a2cd-79ba54be9383 None admin] PATH_INFO: `/v3/services/image` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:56.586127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73420107-5391-402e-a2cd-79ba54be9383 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:56.587519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73420107-5391-402e-a2cd-79ba54be9383 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-73420107-5391-402e-a2cd-79ba54be9383 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:56.598937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:56.601728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:56.602651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 523/1061] 10.222.0.79 () {56 vars in 1105 bytes} [Mon Jul 4 09:56:56 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:56.603468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73420107-5391-402e-a2cd-79ba54be9383 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:56.621445 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ba562-75f9-43b5-a05f-d024eb7c62f3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:56.654222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953a194-5928-4386-9970-e0eea1b6e2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:56.654993 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953a194-5928-4386-9970-e0eea1b6e2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:56.655271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953a194-5928-4386-9970-e0eea1b6e2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:56.655549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953a194-5928-4386-9970-e0eea1b6e2f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:56.656356 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953a194-5928-4386-9970-e0eea1b6e2f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:56.656646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953a194-5928-4386-9970-e0eea1b6e2f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:56.658149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953a194-5928-4386-9970-e0eea1b6e2f4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:56.670340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 539/1062] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:56:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:56.671139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953a194-5928-4386-9970-e0eea1b6e2f4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:56.701576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73420107-5391-402e-a2cd-79ba54be9383 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:56.734682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc3602-7849-4a39-a18f-9cdb31f1bee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:56.735470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc3602-7849-4a39-a18f-9cdb31f1bee2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:56.735748 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc3602-7849-4a39-a18f-9cdb31f1bee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:56.736018 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc3602-7849-4a39-a18f-9cdb31f1bee2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:56.737840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:56.746614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc3602-7849-4a39-a18f-9cdb31f1bee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-c5dc3602-7849-4a39-a18f-9cdb31f1bee2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:56.754427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:56.755591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 524/1063] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:56:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:56.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc3602-7849-4a39-a18f-9cdb31f1bee2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:56.772738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953a194-5928-4386-9970-e0eea1b6e2f4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:56.775129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:56.808106 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e3d80-4cfd-4e95-b380-f0bc7b512d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:56.808702 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e3d80-4cfd-4e95-b380-f0bc7b512d90 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:56.808985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e3d80-4cfd-4e95-b380-f0bc7b512d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:56.809262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e3d80-4cfd-4e95-b380-f0bc7b512d90 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:56.809999 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e3d80-4cfd-4e95-b380-f0bc7b512d90 None admin] RBAC: Adding query filter params (name=image) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:56.810303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e3d80-4cfd-4e95-b380-f0bc7b512d90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:56.811797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e3d80-4cfd-4e95-b380-f0bc7b512d90 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:56.816701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 540/1064] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:56.817555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e3d80-4cfd-4e95-b380-f0bc7b512d90 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:56.858181 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc3602-7849-4a39-a18f-9cdb31f1bee2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:56.893246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f482f-7fb0-4e3f-91c9-00c6e68ca0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:56.894162 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f482f-7fb0-4e3f-91c9-00c6e68ca0b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:56.894449 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f482f-7fb0-4e3f-91c9-00c6e68ca0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:56.894724 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f482f-7fb0-4e3f-91c9-00c6e68ca0b6 None admin] PATH_INFO: `/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/ffc2409cb7fc4373a74ddfb48d501d30/roles/44488839b54a4c9197dca8c9e2226e84` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:56.896957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f482f-7fb0-4e3f-91c9-00c6e68ca0b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f32306262e8744eab4bdc6d28e24e6d0, user_id=ffc2409cb7fc4373a74ddfb48d501d30, role_id=44488839b54a4c9197dca8c9e2226e84)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:56.898427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f482f-7fb0-4e3f-91c9-00c6e68ca0b6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:56.912878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 525/1065] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:56:56 2022] PUT /identity/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/ffc2409cb7fc4373a74ddfb48d501d30/roles/44488839b54a4c9197dca8c9e2226e84 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:56:56.913460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f482f-7fb0-4e3f-91c9-00c6e68ca0b6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:56.930722 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e3d80-4cfd-4e95-b380-f0bc7b512d90 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:56.970448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ade657-ef8a-48a8-864f-b48a5e9aebe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:56.971478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ade657-ef8a-48a8-864f-b48a5e9aebe4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:56.971974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ade657-ef8a-48a8-864f-b48a5e9aebe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:56.972434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ade657-ef8a-48a8-864f-b48a5e9aebe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:56.973389 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ade657-ef8a-48a8-864f-b48a5e9aebe4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:56.973971 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ade657-ef8a-48a8-864f-b48a5e9aebe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:56.975933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ade657-ef8a-48a8-864f-b48a5e9aebe4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:56.976655 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:56.990641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 541/1066] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:56:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:56.992193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ade657-ef8a-48a8-864f-b48a5e9aebe4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:57.027074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:57.036943 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f482f-7fb0-4e3f-91c9-00c6e68ca0b6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:57.077426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd927d7-edf2-4857-b446-e94a227b3ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:57.077919 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd927d7-edf2-4857-b446-e94a227b3ddc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:57.078205 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd927d7-edf2-4857-b446-e94a227b3ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:57.078481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd927d7-edf2-4857-b446-e94a227b3ddc None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:57.079201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd927d7-edf2-4857-b446-e94a227b3ddc None admin] RBAC: Adding query filter params (type=image) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:57.079507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd927d7-edf2-4857-b446-e94a227b3ddc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:57.080917 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd927d7-edf2-4857-b446-e94a227b3ddc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:57.086743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 526/1067] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:57.093514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd927d7-edf2-4857-b446-e94a227b3ddc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:57.116590 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ade657-ef8a-48a8-864f-b48a5e9aebe4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:57.158533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e317b-32dc-45e4-b6fe-41404b77f939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:57.159417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e317b-32dc-45e4-b6fe-41404b77f939 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:57.159701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e317b-32dc-45e4-b6fe-41404b77f939 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:57.159974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e317b-32dc-45e4-b6fe-41404b77f939 None admin] PATH_INFO: `/v3/users/0d9394ce1e664bc692ca4e548258d575` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:57.176034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e317b-32dc-45e4-b6fe-41404b77f939 None admin] RBAC: Authorizing `identity:get_user(user_id=0d9394ce1e664bc692ca4e548258d575)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:57.177532 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e317b-32dc-45e4-b6fe-41404b77f939 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:57.183846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 542/1068] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:56:56 2022] GET /identity/v3/users/0d9394ce1e664bc692ca4e548258d575 => generated 285 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:57.184874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e317b-32dc-45e4-b6fe-41404b77f939 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:57.205180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd927d7-edf2-4857-b446-e94a227b3ddc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:57.222517 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:57.241309 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75047715-9955-4f54-acef-bf7e3e429363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:57.242292 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75047715-9955-4f54-acef-bf7e3e429363 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:57.242571 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75047715-9955-4f54-acef-bf7e3e429363 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:57.242842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75047715-9955-4f54-acef-bf7e3e429363 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:57.243564 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75047715-9955-4f54-acef-bf7e3e429363 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:57.244949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75047715-9955-4f54-acef-bf7e3e429363 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:57.277905 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 527/1069] 10.222.0.79 () {60 vars in 1144 bytes} [Mon Jul 4 09:56:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:56:57.288197 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:57.302497 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e317b-32dc-45e4-b6fe-41404b77f939 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:57.350546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d0d7c-0f83-488a-b981-b236a12eb172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:57.351141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d0d7c-0f83-488a-b981-b236a12eb172 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:57.351477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d0d7c-0f83-488a-b981-b236a12eb172 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:57.351761 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d0d7c-0f83-488a-b981-b236a12eb172 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:57.427440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d0d7c-0f83-488a-b981-b236a12eb172 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:57.428875 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d0d7c-0f83-488a-b981-b236a12eb172 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:57.515177 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:57.538281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 543/1070] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:56:57.539707 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75047715-9955-4f54-acef-bf7e3e429363 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:57.648629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75047715-9955-4f54-acef-bf7e3e429363 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:57.683156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0bf58-50a9-4f01-bd66-40f86364d785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:57.684024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0bf58-50a9-4f01-bd66-40f86364d785 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:57.684307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0bf58-50a9-4f01-bd66-40f86364d785 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:57.684577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0bf58-50a9-4f01-bd66-40f86364d785 None admin] PATH_INFO: `/v3/projects/d50f991d37cd40fca8f218f69a4115c2` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:57.685828 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0bf58-50a9-4f01-bd66-40f86364d785 None admin] RBAC: Authorizing `identity:get_project(project_id=d50f991d37cd40fca8f218f69a4115c2)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:57.687623 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0bf58-50a9-4f01-bd66-40f86364d785 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:57.689497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 528/1071] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:56:57 2022] GET /identity/v3/projects/d50f991d37cd40fca8f218f69a4115c2 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:57.695493 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d0d7c-0f83-488a-b981-b236a12eb172 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:57.801032 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d0d7c-0f83-488a-b981-b236a12eb172 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:57.836243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36378d80-352e-4120-84d9-8a00b33d53c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:57.836912 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36378d80-352e-4120-84d9-8a00b33d53c0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:57.837176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36378d80-352e-4120-84d9-8a00b33d53c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:57.837453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36378d80-352e-4120-84d9-8a00b33d53c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:57.838315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36378d80-352e-4120-84d9-8a00b33d53c0 None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=d50f991d37cd40fca8f218f69a4115c2, user.id=0d9394ce1e664bc692ca4e548258d575) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:57.838625 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36378d80-352e-4120-84d9-8a00b33d53c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:57.840059 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36378d80-352e-4120-84d9-8a00b33d53c0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:57.845893 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 544/1072] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:56:57 2022] GET /identity/v3/role_assignments?user.id=0d9394ce1e664bc692ca4e548258d575&scope.project.id=d50f991d37cd40fca8f218f69a4115c2&role.id=a3de9df253294d069dc1867fa26600c3 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:58.965887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761944a-2dd4-4af8-8a67-feb6c986c15f None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:58.966442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761944a-2dd4-4af8-8a67-feb6c986c15f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:58.966442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761944a-2dd4-4af8-8a67-feb6c986c15f None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:58.967851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 529/1073] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:58.973553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9d2d7-d3ca-4e8b-8111-c603c14fa120 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:58.973998 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9d2d7-d3ca-4e8b-8111-c603c14fa120 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:58.974197 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9d2d7-d3ca-4e8b-8111-c603c14fa120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:58.992613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-63d9d2d7-d3ca-4e8b-8111-c603c14fa120 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:58.993834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-63d9d2d7-d3ca-4e8b-8111-c603c14fa120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:59.040692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:59.067563 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 545/1074] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:56:59.091685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761944a-2dd4-4af8-8a67-feb6c986c15f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:59.149138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:59.212331 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761944a-2dd4-4af8-8a67-feb6c986c15f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:59.249542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345964b-b492-418d-83ec-0487ca07ecaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:59.250637 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345964b-b492-418d-83ec-0487ca07ecaa None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:59.251071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345964b-b492-418d-83ec-0487ca07ecaa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:59.251495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345964b-b492-418d-83ec-0487ca07ecaa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:59.263079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345964b-b492-418d-83ec-0487ca07ecaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-7345964b-b492-418d-83ec-0487ca07ecaa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:56:59.271815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:59.272831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:59.273718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:59.273718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:59.273718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:59.273718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:59.273718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:56:59.273718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:56:59.273718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:56:59.273718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:56:59.273718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:56:59.273718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:56:59.273718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:56:59.273718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:56:59.274368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 530/1075] 10.222.0.79 () {56 vars in 1120 bytes} [Mon Jul 4 09:56:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:59.280079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9d2d7-d3ca-4e8b-8111-c603c14fa120 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:59.359467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1376-ebbc-4b72-9160-26ec11d558c2 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:59.359818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1376-ebbc-4b72-9160-26ec11d558c2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:59.360036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1376-ebbc-4b72-9160-26ec11d558c2 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:59.361585 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 531/1076] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:56:59.366381 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf99938-6dff-45e5-8a24-9c22ef819406 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:59.366675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf99938-6dff-45e5-8a24-9c22ef819406 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:59.366934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf99938-6dff-45e5-8a24-9c22ef819406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:59.383970 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2cf99938-6dff-45e5-8a24-9c22ef819406 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:59.385164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2cf99938-6dff-45e5-8a24-9c22ef819406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:59.387469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9d2d7-d3ca-4e8b-8111-c603c14fa120 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:59.429227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba89db-a945-4737-82ad-44d44401cd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:59.430251 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba89db-a945-4737-82ad-44d44401cd29 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:59.430688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba89db-a945-4737-82ad-44d44401cd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:59.431145 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba89db-a945-4737-82ad-44d44401cd29 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:59.431908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba89db-a945-4737-82ad-44d44401cd29 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:59.432223 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba89db-a945-4737-82ad-44d44401cd29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:59.433654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba89db-a945-4737-82ad-44d44401cd29 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:59.446943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 546/1077] 10.222.0.79 () {56 vars in 1115 bytes} [Mon Jul 4 09:56:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:56:59.448023 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba89db-a945-4737-82ad-44d44401cd29 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:59.473869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 532/1078] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:56:59.495518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:59.498433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf99938-6dff-45e5-8a24-9c22ef819406 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:59.535483 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:59.557456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba89db-a945-4737-82ad-44d44401cd29 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:59.592901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f884084-beb7-471c-ba9d-e0c4ca200f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:59.593746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f884084-beb7-471c-ba9d-e0c4ca200f46 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:59.594035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f884084-beb7-471c-ba9d-e0c4ca200f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:59.594311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f884084-beb7-471c-ba9d-e0c4ca200f46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:59.605236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f884084-beb7-471c-ba9d-e0c4ca200f46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:59.606714 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f884084-beb7-471c-ba9d-e0c4ca200f46 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:59.612706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf99938-6dff-45e5-8a24-9c22ef819406 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-9f884084-beb7-471c-ba9d-e0c4ca200f46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:56:59.624519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:59.625592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:59.626558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:56:59.626558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:56:59.626558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:56:59.626558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:59.630736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 547/1079] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:56:59.631367 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f884084-beb7-471c-ba9d-e0c4ca200f46 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:59.652022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a2c95-7427-430a-b9e5-d4f639099d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:59.652681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a2c95-7427-430a-b9e5-d4f639099d58 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:59.652965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a2c95-7427-430a-b9e5-d4f639099d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:59.653247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a2c95-7427-430a-b9e5-d4f639099d58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:59.727129 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a2c95-7427-430a-b9e5-d4f639099d58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:59.728600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a2c95-7427-430a-b9e5-d4f639099d58 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:59.731892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:59.742364 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f884084-beb7-471c-ba9d-e0c4ca200f46 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:59.762895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:59.780710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb3878-9f27-4ac2-9194-f2ad1322d521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:59.781679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb3878-9f27-4ac2-9194-f2ad1322d521 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:59.782111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb3878-9f27-4ac2-9194-f2ad1322d521 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:59.782574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb3878-9f27-4ac2-9194-f2ad1322d521 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:59.783459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb3878-9f27-4ac2-9194-f2ad1322d521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:56:59.783942 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb3878-9f27-4ac2-9194-f2ad1322d521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:59.785602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb3878-9f27-4ac2-9194-f2ad1322d521 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:59.798816 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 548/1080] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:56:59.804678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb3878-9f27-4ac2-9194-f2ad1322d521 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:59.844671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 533/1081] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:56:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 09:56:59.846050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce71a3-e544-464c-890a-2017238fdf01 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:59.846340 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce71a3-e544-464c-890a-2017238fdf01 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:59.846615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce71a3-e544-464c-890a-2017238fdf01 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:59.847994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 534/1082] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:56:59.850800 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce71a3-e544-464c-890a-2017238fdf01 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:56:59.909776 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb3878-9f27-4ac2-9194-f2ad1322d521 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:59.945802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627763e6-3776-42cc-aa37-f21958aa7139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:59.946611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627763e6-3776-42cc-aa37-f21958aa7139 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:59.946890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627763e6-3776-42cc-aa37-f21958aa7139 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:59.947150 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627763e6-3776-42cc-aa37-f21958aa7139 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:59.954263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627763e6-3776-42cc-aa37-f21958aa7139 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:56:59.955698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627763e6-3776-42cc-aa37-f21958aa7139 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:56:59.956815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce71a3-e544-464c-890a-2017238fdf01 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-627763e6-3776-42cc-aa37-f21958aa7139 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:56:59.970872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:56:59.971832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:56:59.972778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:56:59.972778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:56:59.972778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:56:59.972778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:56:59.972778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:56:59.972778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:56:59.972778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:56:59.972778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:56:59.972778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:56:59.972778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:56:59.972778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 549/1083] 10.222.0.79 () {56 vars in 1093 bytes} [Mon Jul 4 09:56:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:56:59.973669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e15ec-8487-43aa-a538-f7d036772be2 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:59.974186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e15ec-8487-43aa-a538-f7d036772be2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:59.974278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e15ec-8487-43aa-a538-f7d036772be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:56:59.990813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-815e15ec-8487-43aa-a538-f7d036772be2 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:56:59.991392 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:56:59.992365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-815e15ec-8487-43aa-a538-f7d036772be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:56:59.997208 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03feb8-6f15-4128-bb95-717c094efa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:56:59.998034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03feb8-6f15-4128-bb95-717c094efa27 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:56:59.998503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03feb8-6f15-4128-bb95-717c094efa27 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:56:59.998781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03feb8-6f15-4128-bb95-717c094efa27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:00.000585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:00.009248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03feb8-6f15-4128-bb95-717c094efa27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:00.010865 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03feb8-6f15-4128-bb95-717c094efa27 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-ff03feb8-6f15-4128-bb95-717c094efa27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:57:00.028211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:57:00.029134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:57:00.030120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:57:00.030120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:57:00.030120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:57:00.030120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:57:00.030120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 535/1084] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:57:00.030399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03feb8-6f15-4128-bb95-717c094efa27 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:00.065452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 550/1085] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:56:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:00.070608 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e15ec-8487-43aa-a538-f7d036772be2 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:00.141200 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03feb8-6f15-4128-bb95-717c094efa27 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:00.175053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df306b4-d3d0-4ffd-8940-e2ba19037391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:00.175721 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df306b4-d3d0-4ffd-8940-e2ba19037391 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:00.176019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df306b4-d3d0-4ffd-8940-e2ba19037391 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:00.176259 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df306b4-d3d0-4ffd-8940-e2ba19037391 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:00.177090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df306b4-d3d0-4ffd-8940-e2ba19037391 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:00.177336 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e15ec-8487-43aa-a538-f7d036772be2 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:00.177441 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df306b4-d3d0-4ffd-8940-e2ba19037391 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:00.178886 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df306b4-d3d0-4ffd-8940-e2ba19037391 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:00.195947 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 536/1086] 10.222.0.79 () {56 vars in 1133 bytes} [Mon Jul 4 09:57:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:00.196776 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df306b4-d3d0-4ffd-8940-e2ba19037391 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:00.212290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09590b54-0269-40ed-8b0e-b456db8da481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:00.212944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09590b54-0269-40ed-8b0e-b456db8da481 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:00.213241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09590b54-0269-40ed-8b0e-b456db8da481 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:00.213504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09590b54-0269-40ed-8b0e-b456db8da481 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:00.214264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09590b54-0269-40ed-8b0e-b456db8da481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:00.214571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09590b54-0269-40ed-8b0e-b456db8da481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:00.215949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09590b54-0269-40ed-8b0e-b456db8da481 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:00.227915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 551/1087] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:00.228725 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09590b54-0269-40ed-8b0e-b456db8da481 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:00.231222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:00.254612 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:00.317403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df306b4-d3d0-4ffd-8940-e2ba19037391 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:00.347973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09590b54-0269-40ed-8b0e-b456db8da481 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:00.356755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de481bd-a2c9-4574-a436-9a12e5d2afcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:00.357719 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de481bd-a2c9-4574-a436-9a12e5d2afcc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:00.358013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de481bd-a2c9-4574-a436-9a12e5d2afcc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:00.358955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de481bd-a2c9-4574-a436-9a12e5d2afcc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:00.370380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de481bd-a2c9-4574-a436-9a12e5d2afcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-6de481bd-a2c9-4574-a436-9a12e5d2afcc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:57:00.378219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:57:00.379344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:57:00.380247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:57:00.380247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:57:00.380247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:57:00.380247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:57:00.380247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:57:00.380247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:57:00.380247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:57:00.380247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:57:00.380247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:57:00.380247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:57:00.380247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:00.380247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:57:00.380247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 537/1088] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:57:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:57:00.385651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de481bd-a2c9-4574-a436-9a12e5d2afcc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:00.396258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e0772-a7ef-4f26-83b0-74eb290ff251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:00.397170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0772-a7ef-4f26-83b0-74eb290ff251 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:00.397589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0772-a7ef-4f26-83b0-74eb290ff251 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:00.398041 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0772-a7ef-4f26-83b0-74eb290ff251 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:00.470879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e0772-a7ef-4f26-83b0-74eb290ff251 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:00.472356 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e0772-a7ef-4f26-83b0-74eb290ff251 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:00.474527 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:00.518208 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de481bd-a2c9-4574-a436-9a12e5d2afcc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:00.519807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:00.554873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7088804-5081-4fa9-bd90-430b770aaa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:00.555584 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7088804-5081-4fa9-bd90-430b770aaa55 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:00.555839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7088804-5081-4fa9-bd90-430b770aaa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:00.556134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7088804-5081-4fa9-bd90-430b770aaa55 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:00.556876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7088804-5081-4fa9-bd90-430b770aaa55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:00.557178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7088804-5081-4fa9-bd90-430b770aaa55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:00.558590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7088804-5081-4fa9-bd90-430b770aaa55 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:00.566742 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 538/1089] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:57:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:57:00.572569 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7088804-5081-4fa9-bd90-430b770aaa55 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:00.580075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 552/1090] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:57:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:57:00.585805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e0772-a7ef-4f26-83b0-74eb290ff251 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:00.669896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7088804-5081-4fa9-bd90-430b770aaa55 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:00.684721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e0772-a7ef-4f26-83b0-74eb290ff251 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:00.686380 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:00.705060 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bfee7-ee5b-470e-859e-4467fa835f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:00.706001 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bfee7-ee5b-470e-859e-4467fa835f1e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:00.706284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bfee7-ee5b-470e-859e-4467fa835f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:00.706561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bfee7-ee5b-470e-859e-4467fa835f1e None admin] PATH_INFO: `/v3/users/0d9394ce1e664bc692ca4e548258d575` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:00.707774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524bfee7-ee5b-470e-859e-4467fa835f1e None admin] RBAC: Authorizing `identity:get_user(user_id=0d9394ce1e664bc692ca4e548258d575)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:00.709239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524bfee7-ee5b-470e-859e-4467fa835f1e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:00.710942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 539/1091] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:00 2022] GET /identity/v3/users/0d9394ce1e664bc692ca4e548258d575 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:57:00.716536 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bfee7-ee5b-470e-859e-4467fa835f1e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:00.721972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693e602-c137-4f61-a2e3-c1d7f3847a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:00.722773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693e602-c137-4f61-a2e3-c1d7f3847a66 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:00.723055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693e602-c137-4f61-a2e3-c1d7f3847a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:00.723326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693e602-c137-4f61-a2e3-c1d7f3847a66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:00.733474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693e602-c137-4f61-a2e3-c1d7f3847a66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:00.735033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693e602-c137-4f61-a2e3-c1d7f3847a66 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:00.747610 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-5693e602-c137-4f61-a2e3-c1d7f3847a66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:57:00.753526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:57:00.754643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:57:00.755558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:57:00.755558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:57:00.755558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:57:00.755558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:57:00.755558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 553/1092] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:57:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:57:00.760535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693e602-c137-4f61-a2e3-c1d7f3847a66 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:00.818113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bfee7-ee5b-470e-859e-4467fa835f1e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:00.853446 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfed9d4-e9e0-480e-92bf-21c6742d20f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:00.854281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfed9d4-e9e0-480e-92bf-21c6742d20f1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:00.854573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfed9d4-e9e0-480e-92bf-21c6742d20f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:00.854844 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfed9d4-e9e0-480e-92bf-21c6742d20f1 None admin] PATH_INFO: `/v3/projects/d50f991d37cd40fca8f218f69a4115c2` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:00.855983 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfed9d4-e9e0-480e-92bf-21c6742d20f1 None admin] RBAC: Authorizing `identity:get_project(project_id=d50f991d37cd40fca8f218f69a4115c2)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:00.857522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfed9d4-e9e0-480e-92bf-21c6742d20f1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:00.859273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 540/1093] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:57:00 2022] GET /identity/v3/projects/d50f991d37cd40fca8f218f69a4115c2 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:00.859557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693e602-c137-4f61-a2e3-c1d7f3847a66 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:00.865069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfed9d4-e9e0-480e-92bf-21c6742d20f1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:00.893698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e4905-7ef8-47f6-8f37-7c14610f3e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:00.894470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e4905-7ef8-47f6-8f37-7c14610f3e8d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:00.894772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e4905-7ef8-47f6-8f37-7c14610f3e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:00.895046 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e4905-7ef8-47f6-8f37-7c14610f3e8d None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:00.895771 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e4905-7ef8-47f6-8f37-7c14610f3e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:00.896073 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e4905-7ef8-47f6-8f37-7c14610f3e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:00.897445 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e4905-7ef8-47f6-8f37-7c14610f3e8d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:00.913260 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 554/1094] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:57:00.914343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e4905-7ef8-47f6-8f37-7c14610f3e8d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:00.929561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:00.962438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:00.965300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfed9d4-e9e0-480e-92bf-21c6742d20f1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:00.998838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c0635-4d43-4b59-bac4-4b9b990ca87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:00.999720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c0635-4d43-4b59-bac4-4b9b990ca87b None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:01.000042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c0635-4d43-4b59-bac4-4b9b990ca87b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:01.000308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c0635-4d43-4b59-bac4-4b9b990ca87b None admin] PATH_INFO: `/v3/projects/d50f991d37cd40fca8f218f69a4115c2/users/0d9394ce1e664bc692ca4e548258d575/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:01.002350 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c0635-4d43-4b59-bac4-4b9b990ca87b None admin] RBAC: Authorizing `identity:create_grant(project_id=d50f991d37cd40fca8f218f69a4115c2, user_id=0d9394ce1e664bc692ca4e548258d575, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:01.003829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c0635-4d43-4b59-bac4-4b9b990ca87b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:01.011639 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e4905-7ef8-47f6-8f37-7c14610f3e8d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:01.017142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 541/1095] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:57:00 2022] PUT /identity/v3/projects/d50f991d37cd40fca8f218f69a4115c2/users/0d9394ce1e664bc692ca4e548258d575/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:57:01.052638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217948c-0c8d-44c4-8cfe-d0ecad0a40a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:01.053462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217948c-0c8d-44c4-8cfe-d0ecad0a40a0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:01.053814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217948c-0c8d-44c4-8cfe-d0ecad0a40a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:01.054071 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217948c-0c8d-44c4-8cfe-d0ecad0a40a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:01.061910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217948c-0c8d-44c4-8cfe-d0ecad0a40a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:01.063395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217948c-0c8d-44c4-8cfe-d0ecad0a40a0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-8217948c-0c8d-44c4-8cfe-d0ecad0a40a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:57:01.079021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:57:01.080176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:57:01.081810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:57:01.081810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:57:01.081810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:57:01.081810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:57:01.081810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:57:01.081810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:57:01.081810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:57:01.081810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:57:01.081810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 555/1096] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:57:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:57:01.086660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c0635-4d43-4b59-bac4-4b9b990ca87b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:01.198745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c0635-4d43-4b59-bac4-4b9b990ca87b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:01.233561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371c46c-499a-4d4f-bcfc-a40f95284de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:01.234246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371c46c-499a-4d4f-bcfc-a40f95284de3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:01.234525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371c46c-499a-4d4f-bcfc-a40f95284de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:01.234798 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371c46c-499a-4d4f-bcfc-a40f95284de3 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:01.235577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371c46c-499a-4d4f-bcfc-a40f95284de3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:01.235880 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371c46c-499a-4d4f-bcfc-a40f95284de3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:01.237323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371c46c-499a-4d4f-bcfc-a40f95284de3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:01.249528 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 542/1097] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:57:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:57:01.255745 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217948c-0c8d-44c4-8cfe-d0ecad0a40a0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:01.333802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:01.361467 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217948c-0c8d-44c4-8cfe-d0ecad0a40a0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:01.396057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a8fa4-e043-4465-bedc-2731755c4a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:01.396655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a8fa4-e043-4465-bedc-2731755c4a3e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:01.396931 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a8fa4-e043-4465-bedc-2731755c4a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:01.397203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a8fa4-e043-4465-bedc-2731755c4a3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:01.398055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a8fa4-e043-4465-bedc-2731755c4a3e None admin] RBAC: Adding query filter params (role.id=44488839b54a4c9197dca8c9e2226e84, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=ffc2409cb7fc4373a74ddfb48d501d30) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:01.398366 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a8fa4-e043-4465-bedc-2731755c4a3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:01.399798 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a8fa4-e043-4465-bedc-2731755c4a3e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:01.405753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 556/1098] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:57:01 2022] GET /identity/v3/role_assignments?user.id=ffc2409cb7fc4373a74ddfb48d501d30&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=44488839b54a4c9197dca8c9e2226e84 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:03.058447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96884186-24fe-4734-800f-a5a66bf8c54a None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:03.058994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96884186-24fe-4734-800f-a5a66bf8c54a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:03.058994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96884186-24fe-4734-800f-a5a66bf8c54a None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:03.061320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 543/1099] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:03.070144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7226fe-ddb2-46ec-b09a-2502f02a318c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:03.070589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7226fe-ddb2-46ec-b09a-2502f02a318c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:03.070683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7226fe-ddb2-46ec-b09a-2502f02a318c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:03.090585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0e7226fe-ddb2-46ec-b09a-2502f02a318c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:03.091804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0e7226fe-ddb2-46ec-b09a-2502f02a318c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:03.163638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 557/1100] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:03.187786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96884186-24fe-4734-800f-a5a66bf8c54a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:03.213169 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:03.300806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96884186-24fe-4734-800f-a5a66bf8c54a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:03.335330 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747de66-b8e2-46c7-9b3f-dfa94e10b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:03.336043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747de66-b8e2-46c7-9b3f-dfa94e10b342 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:03.336326 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747de66-b8e2-46c7-9b3f-dfa94e10b342 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:03.336603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747de66-b8e2-46c7-9b3f-dfa94e10b342 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:03.347651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747de66-b8e2-46c7-9b3f-dfa94e10b342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-9747de66-b8e2-46c7-9b3f-dfa94e10b342 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:57:03.355502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:57:03.356568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:57:03.357794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:57:03.357794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:57:03.357794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:57:03.357794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:57:03.357794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:57:03.357794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:57:03.357794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:57:03.357794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:57:03.357794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:57:03.357794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:57:03.357794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:03.357794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:57:03.357794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 544/1101] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:57:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:57:03.363190 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7226fe-ddb2-46ec-b09a-2502f02a318c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:03.437701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:03.474851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7226fe-ddb2-46ec-b09a-2502f02a318c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:03.509864 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6e50-1cde-4b89-8dae-61aa0ec9d583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:03.510516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6e50-1cde-4b89-8dae-61aa0ec9d583 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:03.510804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6e50-1cde-4b89-8dae-61aa0ec9d583 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:03.511080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6e50-1cde-4b89-8dae-61aa0ec9d583 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:03.511823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb6e50-1cde-4b89-8dae-61aa0ec9d583 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:03.512132 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb6e50-1cde-4b89-8dae-61aa0ec9d583 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:03.513517 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb6e50-1cde-4b89-8dae-61aa0ec9d583 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:03.522077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 558/1102] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:57:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:03.528827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747de66-b8e2-46c7-9b3f-dfa94e10b342 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:03.625781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:03.645961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747de66-b8e2-46c7-9b3f-dfa94e10b342 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:03.684278 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9b2c4-1ac7-441b-8f5b-0f835c804b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:03.685136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9b2c4-1ac7-441b-8f5b-0f835c804b59 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:03.685415 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9b2c4-1ac7-441b-8f5b-0f835c804b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:03.685805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9b2c4-1ac7-441b-8f5b-0f835c804b59 None admin] PATH_INFO: `/v3/users/0d9394ce1e664bc692ca4e548258d575` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:03.687025 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e9b2c4-1ac7-441b-8f5b-0f835c804b59 None admin] RBAC: Authorizing `identity:get_user(user_id=0d9394ce1e664bc692ca4e548258d575)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:03.688476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e9b2c4-1ac7-441b-8f5b-0f835c804b59 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:03.690134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 545/1103] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:03 2022] GET /identity/v3/users/0d9394ce1e664bc692ca4e548258d575 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:03.699783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6e50-1cde-4b89-8dae-61aa0ec9d583 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:03.813997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6e50-1cde-4b89-8dae-61aa0ec9d583 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:03.857417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:03.864682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d8bdb-da7b-44a8-acaf-c27f0caf827f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:03.865336 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d8bdb-da7b-44a8-acaf-c27f0caf827f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:03.865646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d8bdb-da7b-44a8-acaf-c27f0caf827f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:03.865936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d8bdb-da7b-44a8-acaf-c27f0caf827f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:03.935856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d8bdb-da7b-44a8-acaf-c27f0caf827f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:03.937185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d8bdb-da7b-44a8-acaf-c27f0caf827f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:04.050185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 559/1104] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:57:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:57:04.059527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9b2c4-1ac7-441b-8f5b-0f835c804b59 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:04.171033 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9b2c4-1ac7-441b-8f5b-0f835c804b59 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:04.206447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d0f0b-f4b7-42cf-af4c-6bec068a34ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:04.207235 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d0f0b-f4b7-42cf-af4c-6bec068a34ec None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:04.207517 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d0f0b-f4b7-42cf-af4c-6bec068a34ec None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:04.207790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d0f0b-f4b7-42cf-af4c-6bec068a34ec None admin] PATH_INFO: `/v3/projects/d50f991d37cd40fca8f218f69a4115c2` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:04.209013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d0f0b-f4b7-42cf-af4c-6bec068a34ec None admin] RBAC: Authorizing `identity:get_project(project_id=d50f991d37cd40fca8f218f69a4115c2)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:04.210512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d0f0b-f4b7-42cf-af4c-6bec068a34ec None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:04.212230 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 546/1105] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:57:04 2022] GET /identity/v3/projects/d50f991d37cd40fca8f218f69a4115c2 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:04.220343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d8bdb-da7b-44a8-acaf-c27f0caf827f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:04.255658 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:04.331259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d8bdb-da7b-44a8-acaf-c27f0caf827f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:04.369888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8f162-c7c1-4c6f-aa4d-82b89b610862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:04.370538 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8f162-c7c1-4c6f-aa4d-82b89b610862 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:04.370818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8f162-c7c1-4c6f-aa4d-82b89b610862 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:04.371094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8f162-c7c1-4c6f-aa4d-82b89b610862 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:04.371916 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8f162-c7c1-4c6f-aa4d-82b89b610862 None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=d50f991d37cd40fca8f218f69a4115c2, user.id=0d9394ce1e664bc692ca4e548258d575) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:04.372228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8f162-c7c1-4c6f-aa4d-82b89b610862 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:04.373811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8f162-c7c1-4c6f-aa4d-82b89b610862 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:04.379583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 560/1106] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:57:04 2022] GET /identity/v3/role_assignments?user.id=0d9394ce1e664bc692ca4e548258d575&scope.project.id=d50f991d37cd40fca8f218f69a4115c2&role.id=a3de9df253294d069dc1867fa26600c3 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:06.407890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c47784-41f4-4880-8b6b-c65d41daf4d0 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:06.408699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c47784-41f4-4880-8b6b-c65d41daf4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:06.408699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c47784-41f4-4880-8b6b-c65d41daf4d0 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:06.410920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 547/1107] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:06.419601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc068bc7-e482-4e13-801d-95440b937e35 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:06.420041 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc068bc7-e482-4e13-801d-95440b937e35 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:06.420677 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc068bc7-e482-4e13-801d-95440b937e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:06.443729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-cc068bc7-e482-4e13-801d-95440b937e35 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:06.444905 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-cc068bc7-e482-4e13-801d-95440b937e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:06.517746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 561/1108] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:06.542424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c47784-41f4-4880-8b6b-c65d41daf4d0 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:06.563448 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:06.663846 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c47784-41f4-4880-8b6b-c65d41daf4d0 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:06.711320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12a676-36be-4e97-b564-4302acbde57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:06.712125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12a676-36be-4e97-b564-4302acbde57e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:06.712408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12a676-36be-4e97-b564-4302acbde57e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:06.712687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12a676-36be-4e97-b564-4302acbde57e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:06.713995 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12a676-36be-4e97-b564-4302acbde57e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:06.715464 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12a676-36be-4e97-b564-4302acbde57e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:06.717988 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 548/1109] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:57:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:06.726627 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc068bc7-e482-4e13-801d-95440b937e35 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:06.762777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:06.861967 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc068bc7-e482-4e13-801d-95440b937e35 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:06.908163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49a45d-fa62-4422-836e-4ea4ea60382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:06.908860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49a45d-fa62-4422-836e-4ea4ea60382e None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:06.909149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49a45d-fa62-4422-836e-4ea4ea60382e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:06.909440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49a45d-fa62-4422-836e-4ea4ea60382e None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:06.910192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49a45d-fa62-4422-836e-4ea4ea60382e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:06.911647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49a45d-fa62-4422-836e-4ea4ea60382e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:06.934698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 562/1110] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:57:06 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:57:09.118072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d12d29-4806-40eb-baae-fa7cd289204e None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:09.118660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d12d29-4806-40eb-baae-fa7cd289204e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:09.118660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d12d29-4806-40eb-baae-fa7cd289204e None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:09.120020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 549/1111] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:09.127688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98298ea5-3b1f-4288-8673-0cb059d4b536 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:09.127688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98298ea5-3b1f-4288-8673-0cb059d4b536 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:09.127688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98298ea5-3b1f-4288-8673-0cb059d4b536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:09.145017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-98298ea5-3b1f-4288-8673-0cb059d4b536 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:09.146253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-98298ea5-3b1f-4288-8673-0cb059d4b536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:09.215880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 563/1112] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:09.240911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d12d29-4806-40eb-baae-fa7cd289204e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:09.330906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:09.358920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d12d29-4806-40eb-baae-fa7cd289204e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:09.395479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23fe44-8fa9-442c-8897-36ed12376626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:09.396272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23fe44-8fa9-442c-8897-36ed12376626 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:09.396564 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23fe44-8fa9-442c-8897-36ed12376626 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:09.396840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23fe44-8fa9-442c-8897-36ed12376626 None admin] PATH_INFO: `/v3/roles/78a15fc7951d4f28beb57f54dace457d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:09.398457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23fe44-8fa9-442c-8897-36ed12376626 None admin] RBAC: Authorizing `identity:get_role(role_id=78a15fc7951d4f28beb57f54dace457d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:09.399851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23fe44-8fa9-442c-8897-36ed12376626 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:09.401221 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 550/1113] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:09 2022] GET /identity/v3/roles/78a15fc7951d4f28beb57f54dace457d => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:09.407433 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98298ea5-3b1f-4288-8673-0cb059d4b536 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:09.419088 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:09.512683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98298ea5-3b1f-4288-8673-0cb059d4b536 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:09.547688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710070f-ef01-4ed4-bb05-4c477fc2a16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:09.548563 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710070f-ef01-4ed4-bb05-4c477fc2a16e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:09.548845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710070f-ef01-4ed4-bb05-4c477fc2a16e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:09.549117 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710070f-ef01-4ed4-bb05-4c477fc2a16e None admin] PATH_INFO: `/v3/users/80382b88937f40bca83aa1b8ee594647` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:09.562693 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710070f-ef01-4ed4-bb05-4c477fc2a16e None admin] RBAC: Authorizing `identity:get_user(user_id=80382b88937f40bca83aa1b8ee594647)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:09.564156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710070f-ef01-4ed4-bb05-4c477fc2a16e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:09.565969 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 564/1114] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:09 2022] GET /identity/v3/users/80382b88937f40bca83aa1b8ee594647 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:09.574947 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23fe44-8fa9-442c-8897-36ed12376626 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:09.692332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23fe44-8fa9-442c-8897-36ed12376626 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:09.728262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727bca4-f1ef-4d87-9ce1-f0a133daef9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:09.728898 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727bca4-f1ef-4d87-9ce1-f0a133daef9d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:09.729184 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727bca4-f1ef-4d87-9ce1-f0a133daef9d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:09.729462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727bca4-f1ef-4d87-9ce1-f0a133daef9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:09.751347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:09.803593 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727bca4-f1ef-4d87-9ce1-f0a133daef9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:09.805061 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727bca4-f1ef-4d87-9ce1-f0a133daef9d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:09.914792 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 551/1115] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:57:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:57:09.921278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710070f-ef01-4ed4-bb05-4c477fc2a16e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:09.996775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:10.037905 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710070f-ef01-4ed4-bb05-4c477fc2a16e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:10.077577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51190296-467b-4213-bb14-6216bb32e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:10.078472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51190296-467b-4213-bb14-6216bb32e21d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:10.078752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51190296-467b-4213-bb14-6216bb32e21d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:10.079028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51190296-467b-4213-bb14-6216bb32e21d None admin] PATH_INFO: `/v3/projects/d50f991d37cd40fca8f218f69a4115c2` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:10.080326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51190296-467b-4213-bb14-6216bb32e21d None admin] RBAC: Authorizing `identity:get_project(project_id=d50f991d37cd40fca8f218f69a4115c2)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:10.081795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51190296-467b-4213-bb14-6216bb32e21d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:10.083492 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 565/1116] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:57:09 2022] GET /identity/v3/projects/d50f991d37cd40fca8f218f69a4115c2 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:10.093706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727bca4-f1ef-4d87-9ce1-f0a133daef9d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:10.150107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:10.200617 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727bca4-f1ef-4d87-9ce1-f0a133daef9d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:10.236926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105c56a-0aad-41ab-ab5b-af760b26e2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:10.237576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105c56a-0aad-41ab-ab5b-af760b26e2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:10.237900 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105c56a-0aad-41ab-ab5b-af760b26e2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:10.238177 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105c56a-0aad-41ab-ab5b-af760b26e2e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:10.239008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105c56a-0aad-41ab-ab5b-af760b26e2e4 None admin] RBAC: Adding query filter params (role.id=78a15fc7951d4f28beb57f54dace457d, scope.project.id=d50f991d37cd40fca8f218f69a4115c2, user.id=80382b88937f40bca83aa1b8ee594647) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:10.239321 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105c56a-0aad-41ab-ab5b-af760b26e2e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:10.240774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105c56a-0aad-41ab-ab5b-af760b26e2e4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:10.246396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 552/1117] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:57:10 2022] GET /identity/v3/role_assignments?user.id=80382b88937f40bca83aa1b8ee594647&scope.project.id=d50f991d37cd40fca8f218f69a4115c2&role.id=78a15fc7951d4f28beb57f54dace457d => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:12.293150 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc897-e4b2-49e3-9d93-47d292e0b443 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:12.293783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc897-e4b2-49e3-9d93-47d292e0b443 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:12.293783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc897-e4b2-49e3-9d93-47d292e0b443 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:12.295602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 566/1118] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:12.303730 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833d705-6dd3-4c06-a6f8-85f0d5a68c7b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:12.304203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833d705-6dd3-4c06-a6f8-85f0d5a68c7b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:12.304313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833d705-6dd3-4c06-a6f8-85f0d5a68c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:12.322833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e833d705-6dd3-4c06-a6f8-85f0d5a68c7b None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:12.324022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e833d705-6dd3-4c06-a6f8-85f0d5a68c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:12.393962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 553/1119] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:57:12.417411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fc897-e4b2-49e3-9d93-47d292e0b443 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:12.525702 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fc897-e4b2-49e3-9d93-47d292e0b443 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:12.562987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf2f4e-7345-4d09-8744-1d2e7c1236d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:12.563859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf2f4e-7345-4d09-8744-1d2e7c1236d6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:12.564147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf2f4e-7345-4d09-8744-1d2e7c1236d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:12.564422 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf2f4e-7345-4d09-8744-1d2e7c1236d6 None admin] PATH_INFO: `/v3/roles/78a15fc7951d4f28beb57f54dace457d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:12.566185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf2f4e-7345-4d09-8744-1d2e7c1236d6 None admin] RBAC: Authorizing `identity:get_role(role_id=78a15fc7951d4f28beb57f54dace457d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:12.567585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf2f4e-7345-4d09-8744-1d2e7c1236d6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:12.568966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 567/1120] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:12 2022] GET /identity/v3/roles/78a15fc7951d4f28beb57f54dace457d => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:12.575034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833d705-6dd3-4c06-a6f8-85f0d5a68c7b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:12.630109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:12.687943 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833d705-6dd3-4c06-a6f8-85f0d5a68c7b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:12.723231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd993a05-5bac-4d5f-b719-d718020c621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:12.724104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd993a05-5bac-4d5f-b719-d718020c621d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:12.724388 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd993a05-5bac-4d5f-b719-d718020c621d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:12.724666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd993a05-5bac-4d5f-b719-d718020c621d None admin] PATH_INFO: `/v3/users/80382b88937f40bca83aa1b8ee594647` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:12.726252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd993a05-5bac-4d5f-b719-d718020c621d None admin] RBAC: Authorizing `identity:get_user(user_id=80382b88937f40bca83aa1b8ee594647)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:12.727717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd993a05-5bac-4d5f-b719-d718020c621d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:12.729371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 554/1121] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:12 2022] GET /identity/v3/users/80382b88937f40bca83aa1b8ee594647 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:12.736266 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf2f4e-7345-4d09-8744-1d2e7c1236d6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:12.740584 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:12.836040 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf2f4e-7345-4d09-8744-1d2e7c1236d6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:12.871520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9608a0b-3f6d-4cf1-b96f-0264c475755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:12.872378 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9608a0b-3f6d-4cf1-b96f-0264c475755f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:12.872657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9608a0b-3f6d-4cf1-b96f-0264c475755f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:12.872937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9608a0b-3f6d-4cf1-b96f-0264c475755f None admin] PATH_INFO: `/v3/projects/d50f991d37cd40fca8f218f69a4115c2` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:12.874260 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9608a0b-3f6d-4cf1-b96f-0264c475755f None admin] RBAC: Authorizing `identity:get_project(project_id=d50f991d37cd40fca8f218f69a4115c2)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:12.875797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9608a0b-3f6d-4cf1-b96f-0264c475755f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:12.877490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 568/1122] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:57:12 2022] GET /identity/v3/projects/d50f991d37cd40fca8f218f69a4115c2 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:12.883916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd993a05-5bac-4d5f-b719-d718020c621d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:12.997322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd993a05-5bac-4d5f-b719-d718020c621d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:13.011554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:13.035420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657382b-5a06-4e61-bc42-4d0c2110e873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:13.036314 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657382b-5a06-4e61-bc42-4d0c2110e873 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:13.036599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657382b-5a06-4e61-bc42-4d0c2110e873 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:13.036874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657382b-5a06-4e61-bc42-4d0c2110e873 None admin] PATH_INFO: `/v3/projects/d50f991d37cd40fca8f218f69a4115c2/users/80382b88937f40bca83aa1b8ee594647/roles/78a15fc7951d4f28beb57f54dace457d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:13.039496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657382b-5a06-4e61-bc42-4d0c2110e873 None admin] RBAC: Authorizing `identity:create_grant(project_id=d50f991d37cd40fca8f218f69a4115c2, user_id=80382b88937f40bca83aa1b8ee594647, role_id=78a15fc7951d4f28beb57f54dace457d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:13.041114 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657382b-5a06-4e61-bc42-4d0c2110e873 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:13.057631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 555/1123] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:57:12 2022] PUT /identity/v3/projects/d50f991d37cd40fca8f218f69a4115c2/users/80382b88937f40bca83aa1b8ee594647/roles/78a15fc7951d4f28beb57f54dace457d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:57:15.033324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f13f59-f02e-4e96-9c17-758c955e37a7 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:15.033995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f13f59-f02e-4e96-9c17-758c955e37a7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:15.033995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f13f59-f02e-4e96-9c17-758c955e37a7 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:15.035272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 569/1124] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:15.041153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c495291-0b8a-40d0-8fea-c825b9f0bd75 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:15.041952 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c495291-0b8a-40d0-8fea-c825b9f0bd75 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:15.042096 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c495291-0b8a-40d0-8fea-c825b9f0bd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:15.063759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5c495291-0b8a-40d0-8fea-c825b9f0bd75 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:15.064960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5c495291-0b8a-40d0-8fea-c825b9f0bd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:15.139346 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 556/1125] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:15.175898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f13f59-f02e-4e96-9c17-758c955e37a7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:15.255126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:15.287122 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f13f59-f02e-4e96-9c17-758c955e37a7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:15.325815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7629c-b0ef-4cb9-8e13-485fc8fdbff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:15.326706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7629c-b0ef-4cb9-8e13-485fc8fdbff9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:15.326964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7629c-b0ef-4cb9-8e13-485fc8fdbff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:15.327242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7629c-b0ef-4cb9-8e13-485fc8fdbff9 None admin] PATH_INFO: `/v3/roles/78a15fc7951d4f28beb57f54dace457d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:15.329026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7629c-b0ef-4cb9-8e13-485fc8fdbff9 None admin] RBAC: Authorizing `identity:get_role(role_id=78a15fc7951d4f28beb57f54dace457d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:15.330441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7629c-b0ef-4cb9-8e13-485fc8fdbff9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:15.331824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 570/1126] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:15 2022] GET /identity/v3/roles/78a15fc7951d4f28beb57f54dace457d => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:15.338590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c495291-0b8a-40d0-8fea-c825b9f0bd75 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:15.459551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c495291-0b8a-40d0-8fea-c825b9f0bd75 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:15.461889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:15.496327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc05578-95a2-47c2-bc00-d2ab3f37c71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:15.497183 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc05578-95a2-47c2-bc00-d2ab3f37c71e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:15.497466 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc05578-95a2-47c2-bc00-d2ab3f37c71e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:15.497766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc05578-95a2-47c2-bc00-d2ab3f37c71e None admin] PATH_INFO: `/v3/users/80382b88937f40bca83aa1b8ee594647` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:15.498968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc05578-95a2-47c2-bc00-d2ab3f37c71e None admin] RBAC: Authorizing `identity:get_user(user_id=80382b88937f40bca83aa1b8ee594647)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:15.500467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc05578-95a2-47c2-bc00-d2ab3f37c71e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:15.502104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 557/1127] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:15 2022] GET /identity/v3/users/80382b88937f40bca83aa1b8ee594647 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:15.508173 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7629c-b0ef-4cb9-8e13-485fc8fdbff9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:15.617768 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7629c-b0ef-4cb9-8e13-485fc8fdbff9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:15.654241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43cb09-3292-40f5-9916-af1d9c753522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:15.654880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43cb09-3292-40f5-9916-af1d9c753522 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:15.655163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43cb09-3292-40f5-9916-af1d9c753522 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:15.655636 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43cb09-3292-40f5-9916-af1d9c753522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:15.666466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:15.728599 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43cb09-3292-40f5-9916-af1d9c753522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:15.730107 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43cb09-3292-40f5-9916-af1d9c753522 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:15.834226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 571/1128] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:57:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:57:15.841090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc05578-95a2-47c2-bc00-d2ab3f37c71e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:15.961263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc05578-95a2-47c2-bc00-d2ab3f37c71e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:15.997965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d437a31-16f4-4f0c-aeea-dbc45d3b6897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:15.998803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d437a31-16f4-4f0c-aeea-dbc45d3b6897 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:15.999080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d437a31-16f4-4f0c-aeea-dbc45d3b6897 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:15.999680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d437a31-16f4-4f0c-aeea-dbc45d3b6897 None admin] PATH_INFO: `/v3/projects/d50f991d37cd40fca8f218f69a4115c2` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:16.000654 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d437a31-16f4-4f0c-aeea-dbc45d3b6897 None admin] RBAC: Authorizing `identity:get_project(project_id=d50f991d37cd40fca8f218f69a4115c2)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:16.002134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d437a31-16f4-4f0c-aeea-dbc45d3b6897 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:16.003816 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 558/1129] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:57:15 2022] GET /identity/v3/projects/d50f991d37cd40fca8f218f69a4115c2 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:16.010121 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43cb09-3292-40f5-9916-af1d9c753522 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:16.074737 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:16.137641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43cb09-3292-40f5-9916-af1d9c753522 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:16.176948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a8179-13b9-45bb-ae3c-b9114bb8a7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:16.177607 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a8179-13b9-45bb-ae3c-b9114bb8a7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:16.177922 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a8179-13b9-45bb-ae3c-b9114bb8a7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:16.178202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a8179-13b9-45bb-ae3c-b9114bb8a7f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:16.179026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a8179-13b9-45bb-ae3c-b9114bb8a7f6 None admin] RBAC: Adding query filter params (role.id=78a15fc7951d4f28beb57f54dace457d, scope.project.id=d50f991d37cd40fca8f218f69a4115c2, user.id=80382b88937f40bca83aa1b8ee594647) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:16.179336 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a8179-13b9-45bb-ae3c-b9114bb8a7f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:16.180773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a8179-13b9-45bb-ae3c-b9114bb8a7f6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:16.186497 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 572/1130] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:57:16 2022] GET /identity/v3/role_assignments?user.id=80382b88937f40bca83aa1b8ee594647&scope.project.id=d50f991d37cd40fca8f218f69a4115c2&role.id=78a15fc7951d4f28beb57f54dace457d => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:18.265955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceead103-94d7-4363-9e7f-f4a9521c5091 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:18.266831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceead103-94d7-4363-9e7f-f4a9521c5091 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:18.266831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceead103-94d7-4363-9e7f-f4a9521c5091 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:18.269022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 559/1131] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:18.277766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663610e-c924-483f-992c-fd80bdb3b288 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:18.278205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663610e-c924-483f-992c-fd80bdb3b288 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:18.278344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663610e-c924-483f-992c-fd80bdb3b288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:18.302667 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b663610e-c924-483f-992c-fd80bdb3b288 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:18.303860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b663610e-c924-483f-992c-fd80bdb3b288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:18.374496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 573/1132] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:18.410670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceead103-94d7-4363-9e7f-f4a9521c5091 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:18.459943 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:18.529220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceead103-94d7-4363-9e7f-f4a9521c5091 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:18.568017 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c6ea3-fdbf-4815-9125-ac0d1c6cb25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:18.568791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c6ea3-fdbf-4815-9125-ac0d1c6cb25c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:18.569069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c6ea3-fdbf-4815-9125-ac0d1c6cb25c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:18.569343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c6ea3-fdbf-4815-9125-ac0d1c6cb25c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:18.570652 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c6ea3-fdbf-4815-9125-ac0d1c6cb25c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:18.572119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c6ea3-fdbf-4815-9125-ac0d1c6cb25c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:18.573933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 560/1133] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:57:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:18.580644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663610e-c924-483f-992c-fd80bdb3b288 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:18.622758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:18.705807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663610e-c924-483f-992c-fd80bdb3b288 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:18.741470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c6acd-052c-4eae-a0e9-d965981da6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:18.742168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c6acd-052c-4eae-a0e9-d965981da6fe None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:18.742449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c6acd-052c-4eae-a0e9-d965981da6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:18.742728 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c6acd-052c-4eae-a0e9-d965981da6fe None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:18.743438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c6acd-052c-4eae-a0e9-d965981da6fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:18.744892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c6acd-052c-4eae-a0e9-d965981da6fe None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:18.775142 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 574/1134] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:57:18 2022] POST /identity/v3/projects => generated 311 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:57:20.820599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82333a-39d3-4812-8e55-bf3d143768f0 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:20.821494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82333a-39d3-4812-8e55-bf3d143768f0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:20.821494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82333a-39d3-4812-8e55-bf3d143768f0 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:20.823958 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 561/1135] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:20.829771 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bba0b1-87aa-46dc-b24c-5471620cb915 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:20.830222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bba0b1-87aa-46dc-b24c-5471620cb915 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:20.830366 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bba0b1-87aa-46dc-b24c-5471620cb915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:20.849157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-97bba0b1-87aa-46dc-b24c-5471620cb915 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:20.850717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-97bba0b1-87aa-46dc-b24c-5471620cb915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:20.926376 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 575/1136] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:57:20.956092 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82333a-39d3-4812-8e55-bf3d143768f0 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:21.066798 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82333a-39d3-4812-8e55-bf3d143768f0 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:21.076042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:21.109462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864e998-08a5-4b71-b012-56a529390aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:21.110283 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864e998-08a5-4b71-b012-56a529390aaa None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:21.110603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864e998-08a5-4b71-b012-56a529390aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:21.110881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864e998-08a5-4b71-b012-56a529390aaa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:21.112122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864e998-08a5-4b71-b012-56a529390aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:21.113603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864e998-08a5-4b71-b012-56a529390aaa None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:21.115281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 562/1137] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:57:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:21.121201 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bba0b1-87aa-46dc-b24c-5471620cb915 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:21.139563 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:21.229603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bba0b1-87aa-46dc-b24c-5471620cb915 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:21.267094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc36eb-f1be-42b1-b0dd-3f265991a488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:21.267761 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc36eb-f1be-42b1-b0dd-3f265991a488 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:21.268043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc36eb-f1be-42b1-b0dd-3f265991a488 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:21.268326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc36eb-f1be-42b1-b0dd-3f265991a488 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:21.269028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc36eb-f1be-42b1-b0dd-3f265991a488 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:21.270498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc36eb-f1be-42b1-b0dd-3f265991a488 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:21.296873 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 576/1138] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:57:21 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:57:23.366463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbf183-2dbe-49e7-b7bd-0033986a151f None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:23.367044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbf183-2dbe-49e7-b7bd-0033986a151f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:23.367044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbf183-2dbe-49e7-b7bd-0033986a151f None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:23.368418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 563/1139] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:23.377382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe47b3c-b732-453e-bdfa-61578ddd0ac1 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:23.378188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe47b3c-b732-453e-bdfa-61578ddd0ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:23.378345 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe47b3c-b732-453e-bdfa-61578ddd0ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:23.400426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0fe47b3c-b732-453e-bdfa-61578ddd0ac1 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:23.401545 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0fe47b3c-b732-453e-bdfa-61578ddd0ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:23.471283 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 577/1140] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:23.500008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbf183-2dbe-49e7-b7bd-0033986a151f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:23.611803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbf183-2dbe-49e7-b7bd-0033986a151f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:23.647049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73eb53-9747-4eec-b53f-50c15e455a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:23.647818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73eb53-9747-4eec-b53f-50c15e455a56 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:23.648097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73eb53-9747-4eec-b53f-50c15e455a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:23.648365 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73eb53-9747-4eec-b53f-50c15e455a56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:23.659985 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73eb53-9747-4eec-b53f-50c15e455a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8d73eb53-9747-4eec-b53f-50c15e455a56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:57:23.667394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:57:23.668542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:57:23.669743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:57:23.669743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:57:23.669743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:57:23.669743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:57:23.669743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:57:23.669743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:57:23.669743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:57:23.669743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:57:23.669743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:57:23.669743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:57:23.669743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:23.669743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:57:23.669743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 564/1141] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:57:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:57:23.675489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe47b3c-b732-453e-bdfa-61578ddd0ac1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:23.678777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:23.783784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe47b3c-b732-453e-bdfa-61578ddd0ac1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:23.826351 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4297104-f3ae-4f55-b2d5-33c7055adc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:23.826985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4297104-f3ae-4f55-b2d5-33c7055adc4c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:23.827349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4297104-f3ae-4f55-b2d5-33c7055adc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:23.827811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4297104-f3ae-4f55-b2d5-33c7055adc4c None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:23.828678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4297104-f3ae-4f55-b2d5-33c7055adc4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:23.828981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4297104-f3ae-4f55-b2d5-33c7055adc4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:23.830386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4297104-f3ae-4f55-b2d5-33c7055adc4c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:23.838332 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 578/1142] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:57:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:23.855031 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73eb53-9747-4eec-b53f-50c15e455a56 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:23.899327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:23.966325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73eb53-9747-4eec-b53f-50c15e455a56 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:24.004662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71272737-7db8-4c23-8d4f-ead27c0966fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:24.005530 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71272737-7db8-4c23-8d4f-ead27c0966fd None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:24.005843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71272737-7db8-4c23-8d4f-ead27c0966fd None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:24.006122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71272737-7db8-4c23-8d4f-ead27c0966fd None admin] PATH_INFO: `/v3/users/f01e400996e0445da739246e126e6ee8` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:24.019746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71272737-7db8-4c23-8d4f-ead27c0966fd None admin] RBAC: Authorizing `identity:get_user(user_id=f01e400996e0445da739246e126e6ee8)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:24.021209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71272737-7db8-4c23-8d4f-ead27c0966fd None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:24.022902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 565/1143] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:23 2022] GET /identity/v3/users/f01e400996e0445da739246e126e6ee8 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:24.029100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4297104-f3ae-4f55-b2d5-33c7055adc4c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:24.098158 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:24.143360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4297104-f3ae-4f55-b2d5-33c7055adc4c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:24.178313 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae3e6d-1805-4a74-86c6-7b4dbb3088d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:24.178962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae3e6d-1805-4a74-86c6-7b4dbb3088d7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:24.179249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae3e6d-1805-4a74-86c6-7b4dbb3088d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:24.179508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae3e6d-1805-4a74-86c6-7b4dbb3088d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:24.256508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae3e6d-1805-4a74-86c6-7b4dbb3088d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:24.257910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae3e6d-1805-4a74-86c6-7b4dbb3088d7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:24.320217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:24.362647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 579/1144] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:57:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:57:24.371836 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71272737-7db8-4c23-8d4f-ead27c0966fd None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:24.468464 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:24.476781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71272737-7db8-4c23-8d4f-ead27c0966fd None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:24.514840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44015625-7771-4d52-a339-56bb69579de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:24.515618 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44015625-7771-4d52-a339-56bb69579de5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:24.515895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44015625-7771-4d52-a339-56bb69579de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:24.516169 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44015625-7771-4d52-a339-56bb69579de5 None admin] PATH_INFO: `/v3/projects/3bb6d5e5d3d540eea4e4293941bf8279` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:24.517398 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44015625-7771-4d52-a339-56bb69579de5 None admin] RBAC: Authorizing `identity:get_project(project_id=3bb6d5e5d3d540eea4e4293941bf8279)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:24.518896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44015625-7771-4d52-a339-56bb69579de5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:24.521895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 566/1145] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:57:24 2022] GET /identity/v3/projects/3bb6d5e5d3d540eea4e4293941bf8279 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:24.528888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae3e6d-1805-4a74-86c6-7b4dbb3088d7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:24.644655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae3e6d-1805-4a74-86c6-7b4dbb3088d7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:24.680443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a468847-b2da-435c-81e7-f64b7ef97832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:24.681093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a468847-b2da-435c-81e7-f64b7ef97832 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:24.681376 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a468847-b2da-435c-81e7-f64b7ef97832 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:24.681662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a468847-b2da-435c-81e7-f64b7ef97832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:24.682502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a468847-b2da-435c-81e7-f64b7ef97832 None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=3bb6d5e5d3d540eea4e4293941bf8279, user.id=f01e400996e0445da739246e126e6ee8) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:24.682814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a468847-b2da-435c-81e7-f64b7ef97832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:24.684253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a468847-b2da-435c-81e7-f64b7ef97832 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:24.690286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 580/1146] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:57:24 2022] GET /identity/v3/role_assignments?user.id=f01e400996e0445da739246e126e6ee8&scope.project.id=3bb6d5e5d3d540eea4e4293941bf8279&role.id=a3de9df253294d069dc1867fa26600c3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:26.680422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd0073-53bf-4b1d-995c-a66c946a6fd0 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:26.680989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd0073-53bf-4b1d-995c-a66c946a6fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:26.680989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd0073-53bf-4b1d-995c-a66c946a6fd0 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:26.682371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 567/1147] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:26.688812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d54c7f-e294-4a96-b122-9e193e5ef21d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:26.689496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d54c7f-e294-4a96-b122-9e193e5ef21d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:26.689676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d54c7f-e294-4a96-b122-9e193e5ef21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:26.716844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-99d54c7f-e294-4a96-b122-9e193e5ef21d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:26.718473 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-99d54c7f-e294-4a96-b122-9e193e5ef21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:26.746499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:26.796320 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 581/1148] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:26.825888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd0073-53bf-4b1d-995c-a66c946a6fd0 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:26.932127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd0073-53bf-4b1d-995c-a66c946a6fd0 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:26.969591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850c6c0-63ef-49cc-950e-90373fd386bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:26.970408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850c6c0-63ef-49cc-950e-90373fd386bc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:26.970692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850c6c0-63ef-49cc-950e-90373fd386bc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:26.970971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850c6c0-63ef-49cc-950e-90373fd386bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:26.983374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850c6c0-63ef-49cc-950e-90373fd386bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1850c6c0-63ef-49cc-950e-90373fd386bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:57:26.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:57:26.992030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:57:26.993151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:57:26.993151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:57:26.993151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:57:26.993151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:57:26.993151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:57:26.993151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:57:26.993151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:57:26.993151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:57:26.993151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:57:26.993151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:57:26.993151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:26.993151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:57:26.993151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 568/1149] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:57:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:57:26.998663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d54c7f-e294-4a96-b122-9e193e5ef21d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:27.101734 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d54c7f-e294-4a96-b122-9e193e5ef21d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:27.141403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b26e4a-06eb-487f-9708-f56b095c12b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:27.142078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b26e4a-06eb-487f-9708-f56b095c12b1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:27.142362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b26e4a-06eb-487f-9708-f56b095c12b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:27.142629 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b26e4a-06eb-487f-9708-f56b095c12b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:27.143391 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b26e4a-06eb-487f-9708-f56b095c12b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:27.143699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b26e4a-06eb-487f-9708-f56b095c12b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:27.145082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b26e4a-06eb-487f-9708-f56b095c12b1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:27.153838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 582/1150] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:57:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:57:27.163270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850c6c0-63ef-49cc-950e-90373fd386bc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:27.193294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:27.275050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850c6c0-63ef-49cc-950e-90373fd386bc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:27.313093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd133e9-9db7-4532-8e88-e69c6617039b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:27.313990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd133e9-9db7-4532-8e88-e69c6617039b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:27.314270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd133e9-9db7-4532-8e88-e69c6617039b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:27.314545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd133e9-9db7-4532-8e88-e69c6617039b None admin] PATH_INFO: `/v3/users/f01e400996e0445da739246e126e6ee8` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:27.315842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd133e9-9db7-4532-8e88-e69c6617039b None admin] RBAC: Authorizing `identity:get_user(user_id=f01e400996e0445da739246e126e6ee8)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:27.317308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd133e9-9db7-4532-8e88-e69c6617039b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:27.318975 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 569/1151] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:27 2022] GET /identity/v3/users/f01e400996e0445da739246e126e6ee8 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:27.325175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b26e4a-06eb-487f-9708-f56b095c12b1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:27.350281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:27.439772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b26e4a-06eb-487f-9708-f56b095c12b1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:27.474993 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5c21d-86b5-4955-916f-8804df09d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:27.475795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5c21d-86b5-4955-916f-8804df09d128 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:27.476074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5c21d-86b5-4955-916f-8804df09d128 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:27.476346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5c21d-86b5-4955-916f-8804df09d128 None admin] PATH_INFO: `/v3/projects/3bb6d5e5d3d540eea4e4293941bf8279` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:27.477670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5c21d-86b5-4955-916f-8804df09d128 None admin] RBAC: Authorizing `identity:get_project(project_id=3bb6d5e5d3d540eea4e4293941bf8279)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:27.479119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5c21d-86b5-4955-916f-8804df09d128 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:27.480807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 583/1152] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:57:27 2022] GET /identity/v3/projects/3bb6d5e5d3d540eea4e4293941bf8279 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:27.491981 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd133e9-9db7-4532-8e88-e69c6617039b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:27.590702 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:27.607715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd133e9-9db7-4532-8e88-e69c6617039b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:27.649843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7bc15-4c72-46bd-a940-76d017365c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:27.650728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7bc15-4c72-46bd-a940-76d017365c21 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:27.651005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7bc15-4c72-46bd-a940-76d017365c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:27.651280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7bc15-4c72-46bd-a940-76d017365c21 None admin] PATH_INFO: `/v3/projects/3bb6d5e5d3d540eea4e4293941bf8279/users/f01e400996e0445da739246e126e6ee8/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:27.653325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7bc15-4c72-46bd-a940-76d017365c21 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bb6d5e5d3d540eea4e4293941bf8279, user_id=f01e400996e0445da739246e126e6ee8, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:27.654811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7bc15-4c72-46bd-a940-76d017365c21 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:27.671254 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 570/1153] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:57:27 2022] PUT /identity/v3/projects/3bb6d5e5d3d540eea4e4293941bf8279/users/f01e400996e0445da739246e126e6ee8/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:57:29.703560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34ad49-8d5c-4df4-be8d-5ca3eb95ec1d None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:29.704673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34ad49-8d5c-4df4-be8d-5ca3eb95ec1d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:29.704673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34ad49-8d5c-4df4-be8d-5ca3eb95ec1d None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:29.706747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 584/1154] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:29.712934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5934e01-1a61-42bb-b0bc-7e5f9bb6867d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:29.713660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5934e01-1a61-42bb-b0bc-7e5f9bb6867d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:29.713815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5934e01-1a61-42bb-b0bc-7e5f9bb6867d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:29.735555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d5934e01-1a61-42bb-b0bc-7e5f9bb6867d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:29.736760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d5934e01-1a61-42bb-b0bc-7e5f9bb6867d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:29.809375 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 571/1155] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:29.833404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34ad49-8d5c-4df4-be8d-5ca3eb95ec1d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:29.936826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:29.956206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34ad49-8d5c-4df4-be8d-5ca3eb95ec1d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:29.990839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d89f5-6e31-4a93-8c41-299cd5846f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:29.991631 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d89f5-6e31-4a93-8c41-299cd5846f4f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:29.991914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d89f5-6e31-4a93-8c41-299cd5846f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:29.992189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d89f5-6e31-4a93-8c41-299cd5846f4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:30.003259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d89f5-6e31-4a93-8c41-299cd5846f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-cd9d89f5-6e31-4a93-8c41-299cd5846f4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:57:30.010975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:57:30.011940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:57:30.012876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:57:30.012876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:57:30.012876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:57:30.012876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:57:30.012876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:57:30.012876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:57:30.012876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:57:30.012876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:57:30.012876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:57:30.012876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:57:30.012876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:30.012876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:57:30.012876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 585/1156] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:57:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:57:30.018896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5934e01-1a61-42bb-b0bc-7e5f9bb6867d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:30.107763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:30.128285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5934e01-1a61-42bb-b0bc-7e5f9bb6867d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:30.166628 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcd77c-c031-4e64-b828-ac481459cdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:30.167292 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcd77c-c031-4e64-b828-ac481459cdf8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:30.167577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcd77c-c031-4e64-b828-ac481459cdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:30.167855 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcd77c-c031-4e64-b828-ac481459cdf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:30.168601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcd77c-c031-4e64-b828-ac481459cdf8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:30.168907 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcd77c-c031-4e64-b828-ac481459cdf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:30.170323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcd77c-c031-4e64-b828-ac481459cdf8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:30.178286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 572/1157] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:57:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:30.186958 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d89f5-6e31-4a93-8c41-299cd5846f4f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:30.306379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d89f5-6e31-4a93-8c41-299cd5846f4f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:30.342623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25890a-ff15-4b51-9d6b-0711feeec863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:30.343494 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25890a-ff15-4b51-9d6b-0711feeec863 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:30.343775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25890a-ff15-4b51-9d6b-0711feeec863 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:30.344052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25890a-ff15-4b51-9d6b-0711feeec863 None admin] PATH_INFO: `/v3/users/f01e400996e0445da739246e126e6ee8` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:30.345546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25890a-ff15-4b51-9d6b-0711feeec863 None admin] RBAC: Authorizing `identity:get_user(user_id=f01e400996e0445da739246e126e6ee8)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:30.347028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25890a-ff15-4b51-9d6b-0711feeec863 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:30.348671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 586/1158] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:30 2022] GET /identity/v3/users/f01e400996e0445da739246e126e6ee8 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:30.355341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcd77c-c031-4e64-b828-ac481459cdf8 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:30.468641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcd77c-c031-4e64-b828-ac481459cdf8 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:30.504319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4107f8b-5246-4fa5-9382-f4a64296b47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:30.504957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4107f8b-5246-4fa5-9382-f4a64296b47e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:30.505241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4107f8b-5246-4fa5-9382-f4a64296b47e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:30.505513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4107f8b-5246-4fa5-9382-f4a64296b47e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:30.517434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:30.594116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4107f8b-5246-4fa5-9382-f4a64296b47e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:30.595537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4107f8b-5246-4fa5-9382-f4a64296b47e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:30.707975 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 573/1159] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:57:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:57:30.714347 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25890a-ff15-4b51-9d6b-0711feeec863 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:30.736268 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:30.832153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25890a-ff15-4b51-9d6b-0711feeec863 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:30.867381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71155d1-7006-45ec-b06c-964e5343a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:30.868172 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71155d1-7006-45ec-b06c-964e5343a16d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:30.868457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71155d1-7006-45ec-b06c-964e5343a16d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:30.868728 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71155d1-7006-45ec-b06c-964e5343a16d None admin] PATH_INFO: `/v3/projects/3bb6d5e5d3d540eea4e4293941bf8279` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:30.869954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71155d1-7006-45ec-b06c-964e5343a16d None admin] RBAC: Authorizing `identity:get_project(project_id=3bb6d5e5d3d540eea4e4293941bf8279)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:30.871398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71155d1-7006-45ec-b06c-964e5343a16d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:30.873097 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 587/1160] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:57:30 2022] GET /identity/v3/projects/3bb6d5e5d3d540eea4e4293941bf8279 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:30.883708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4107f8b-5246-4fa5-9382-f4a64296b47e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:30.933664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:30.995280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4107f8b-5246-4fa5-9382-f4a64296b47e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:31.035057 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667fb76-1101-4fe9-9b68-9107b38fff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:31.035722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667fb76-1101-4fe9-9b68-9107b38fff7b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:31.036003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667fb76-1101-4fe9-9b68-9107b38fff7b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:31.036281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667fb76-1101-4fe9-9b68-9107b38fff7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:31.037118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667fb76-1101-4fe9-9b68-9107b38fff7b None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=3bb6d5e5d3d540eea4e4293941bf8279, user.id=f01e400996e0445da739246e126e6ee8) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:31.037434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667fb76-1101-4fe9-9b68-9107b38fff7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:31.038939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667fb76-1101-4fe9-9b68-9107b38fff7b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:31.045052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 574/1161] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:57:30 2022] GET /identity/v3/role_assignments?user.id=f01e400996e0445da739246e126e6ee8&scope.project.id=3bb6d5e5d3d540eea4e4293941bf8279&role.id=a3de9df253294d069dc1867fa26600c3 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:33.124918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487ccd5-b2fa-4185-8510-37f44ed233d5 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:33.125815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487ccd5-b2fa-4185-8510-37f44ed233d5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:33.125977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487ccd5-b2fa-4185-8510-37f44ed233d5 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:33.127330 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 588/1162] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:33.135999 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9f02e-f39a-4d87-8209-7ad70a8a066a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:33.136735 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9f02e-f39a-4d87-8209-7ad70a8a066a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:33.136897 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9f02e-f39a-4d87-8209-7ad70a8a066a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:33.156371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-9fd9f02e-f39a-4d87-8209-7ad70a8a066a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:33.157550 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9fd9f02e-f39a-4d87-8209-7ad70a8a066a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:33.228178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 575/1163] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:33.251237 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487ccd5-b2fa-4185-8510-37f44ed233d5 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:33.343155 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:33.360923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487ccd5-b2fa-4185-8510-37f44ed233d5 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:33.395471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d07b02-a458-4e72-91e5-3b05ff61f7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:33.396051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d07b02-a458-4e72-91e5-3b05ff61f7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:33.396337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d07b02-a458-4e72-91e5-3b05ff61f7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:33.396613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d07b02-a458-4e72-91e5-3b05ff61f7d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:33.469580 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d07b02-a458-4e72-91e5-3b05ff61f7d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:33.471197 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d07b02-a458-4e72-91e5-3b05ff61f7d7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:33.555134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:33.572814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 589/1164] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:57:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:57:33.579071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9f02e-f39a-4d87-8209-7ad70a8a066a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:33.616163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:33.686022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9f02e-f39a-4d87-8209-7ad70a8a066a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:33.721752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793e220-a065-4826-b589-b0e98aad93aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:33.722545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793e220-a065-4826-b589-b0e98aad93aa None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:33.722826 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793e220-a065-4826-b589-b0e98aad93aa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:33.723100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793e220-a065-4826-b589-b0e98aad93aa None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:33.730586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793e220-a065-4826-b589-b0e98aad93aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:33.732052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793e220-a065-4826-b589-b0e98aad93aa None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8793e220-a065-4826-b589-b0e98aad93aa None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:57:33.746526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:57:33.747708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:57:33.748722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 576/1165] 10.222.0.79 () {56 vars in 1117 bytes} [Mon Jul 4 09:57:33 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:57:33.754672 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d07b02-a458-4e72-91e5-3b05ff61f7d7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:33.867608 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d07b02-a458-4e72-91e5-3b05ff61f7d7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:33.913207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78591a-0b20-467b-ab19-3e33f2ddb1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:33.913899 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78591a-0b20-467b-ab19-3e33f2ddb1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:33.914185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78591a-0b20-467b-ab19-3e33f2ddb1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:33.914458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78591a-0b20-467b-ab19-3e33f2ddb1d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:33.915177 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78591a-0b20-467b-ab19-3e33f2ddb1d3 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:33.915489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78591a-0b20-467b-ab19-3e33f2ddb1d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:33.916863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78591a-0b20-467b-ab19-3e33f2ddb1d3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:33.924885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 590/1166] 10.222.0.79 () {56 vars in 1115 bytes} [Mon Jul 4 09:57:33 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:57:35.938116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eca27f-22dd-4e0e-83af-9a7125d397a9 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:35.939005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eca27f-22dd-4e0e-83af-9a7125d397a9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:35.939005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eca27f-22dd-4e0e-83af-9a7125d397a9 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:35.941141 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 577/1167] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:35.949942 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce45ec4-d3be-4a0f-af89-103d97d98108 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:35.950359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce45ec4-d3be-4a0f-af89-103d97d98108 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:35.950548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce45ec4-d3be-4a0f-af89-103d97d98108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:35.970212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5ce45ec4-d3be-4a0f-af89-103d97d98108 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:35.971408 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5ce45ec4-d3be-4a0f-af89-103d97d98108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:36.021699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:36.061243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 591/1168] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:36.087172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eca27f-22dd-4e0e-83af-9a7125d397a9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:36.193266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:36.204495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eca27f-22dd-4e0e-83af-9a7125d397a9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:36.240142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f037a5-2626-4a33-848c-26f1de834929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:36.240800 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f037a5-2626-4a33-848c-26f1de834929 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:36.241084 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f037a5-2626-4a33-848c-26f1de834929 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:36.241360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f037a5-2626-4a33-848c-26f1de834929 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:36.242135 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f037a5-2626-4a33-848c-26f1de834929 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:36.243526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f037a5-2626-4a33-848c-26f1de834929 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:36.266913 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 578/1169] 10.222.0.79 () {60 vars in 1138 bytes} [Mon Jul 4 09:57:36 2022] POST /identity/v3/domains => generated 267 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:57:38.285384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68d6fa-0dbd-4674-a159-d724aa62ce86 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:38.286289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68d6fa-0dbd-4674-a159-d724aa62ce86 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:38.286559 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68d6fa-0dbd-4674-a159-d724aa62ce86 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:38.288738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 592/1170] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:38.296875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b68c98-b16f-4229-9ba7-ea8d7d57a915 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:38.297340 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b68c98-b16f-4229-9ba7-ea8d7d57a915 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:38.297432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b68c98-b16f-4229-9ba7-ea8d7d57a915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:38.314825 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-51b68c98-b16f-4229-9ba7-ea8d7d57a915 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:38.315996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-51b68c98-b16f-4229-9ba7-ea8d7d57a915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:38.400336 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 579/1171] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:57:38.432776 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68d6fa-0dbd-4674-a159-d724aa62ce86 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:38.541153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68d6fa-0dbd-4674-a159-d724aa62ce86 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:38.575793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277464cc-2fbb-48a5-9f95-08fde257e803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:38.576630 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277464cc-2fbb-48a5-9f95-08fde257e803 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:38.576907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277464cc-2fbb-48a5-9f95-08fde257e803 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:38.577175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277464cc-2fbb-48a5-9f95-08fde257e803 None admin] PATH_INFO: `/v3/domains/ef69dd28c8dc4324bd984d8c9f1f6bb1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:38.589264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277464cc-2fbb-48a5-9f95-08fde257e803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ef69dd28c8dc4324bd984d8c9f1f6bb1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:38.590737 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277464cc-2fbb-48a5-9f95-08fde257e803 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:38.592372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 593/1172] 10.222.0.79 () {56 vars in 1183 bytes} [Mon Jul 4 09:57:38 2022] GET /identity/v3/domains/ef69dd28c8dc4324bd984d8c9f1f6bb1 => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:38.599171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b68c98-b16f-4229-9ba7-ea8d7d57a915 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:38.699434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:38.716566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b68c98-b16f-4229-9ba7-ea8d7d57a915 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:38.752991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694643a-a497-43d7-8e5a-d32783f7f328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:38.753656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694643a-a497-43d7-8e5a-d32783f7f328 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:38.753962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694643a-a497-43d7-8e5a-d32783f7f328 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:38.754242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694643a-a497-43d7-8e5a-d32783f7f328 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:38.754947 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694643a-a497-43d7-8e5a-d32783f7f328 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:38.756398 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694643a-a497-43d7-8e5a-d32783f7f328 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:38.784939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 580/1173] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:57:38 2022] POST /identity/v3/projects => generated 361 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:57:40.835827 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b922678-1f9c-47b5-8b3b-def1168b8c6a None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:40.836411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b922678-1f9c-47b5-8b3b-def1168b8c6a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:40.836411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b922678-1f9c-47b5-8b3b-def1168b8c6a None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:40.837795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 594/1174] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:40.844131 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc97bf-cd79-4d2f-9596-78b166882c55 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:40.844996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc97bf-cd79-4d2f-9596-78b166882c55 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:40.845154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc97bf-cd79-4d2f-9596-78b166882c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:40.873426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4adc97bf-cd79-4d2f-9596-78b166882c55 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:40.875406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4adc97bf-cd79-4d2f-9596-78b166882c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:40.945728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 581/1175] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:40.971115 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b922678-1f9c-47b5-8b3b-def1168b8c6a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:41.017035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:41.090476 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b922678-1f9c-47b5-8b3b-def1168b8c6a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:41.129372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c90622-922e-48ce-afeb-af3e86223263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:41.130271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c90622-922e-48ce-afeb-af3e86223263 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:41.130554 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c90622-922e-48ce-afeb-af3e86223263 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:41.130829 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c90622-922e-48ce-afeb-af3e86223263 None admin] PATH_INFO: `/v3/domains/ef69dd28c8dc4324bd984d8c9f1f6bb1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:41.132087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c90622-922e-48ce-afeb-af3e86223263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ef69dd28c8dc4324bd984d8c9f1f6bb1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:41.133592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c90622-922e-48ce-afeb-af3e86223263 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:41.135279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 595/1176] 10.222.0.79 () {56 vars in 1183 bytes} [Mon Jul 4 09:57:40 2022] GET /identity/v3/domains/ef69dd28c8dc4324bd984d8c9f1f6bb1 => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:41.144303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc97bf-cd79-4d2f-9596-78b166882c55 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:41.227997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:41.263003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc97bf-cd79-4d2f-9596-78b166882c55 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:41.298143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01241a-b068-4dcc-afcc-27a46cf8a06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:41.298803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01241a-b068-4dcc-afcc-27a46cf8a06b None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:41.299086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01241a-b068-4dcc-afcc-27a46cf8a06b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:41.299363 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01241a-b068-4dcc-afcc-27a46cf8a06b None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:41.300071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01241a-b068-4dcc-afcc-27a46cf8a06b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:41.301518 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01241a-b068-4dcc-afcc-27a46cf8a06b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:41.324509 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 582/1177] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:57:41 2022] POST /identity/v3/users => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:57:43.402985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43427b6b-c957-46b8-98d3-3c1d3a302aea None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:43.403826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43427b6b-c957-46b8-98d3-3c1d3a302aea None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:43.403826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43427b6b-c957-46b8-98d3-3c1d3a302aea None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:43.405993 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 596/1178] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:43.412502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfba8d-04b7-493b-bcd4-4d38ff41ddfa None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:43.412930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfba8d-04b7-493b-bcd4-4d38ff41ddfa None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:43.413072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfba8d-04b7-493b-bcd4-4d38ff41ddfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:43.434446 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-86bfba8d-04b7-493b-bcd4-4d38ff41ddfa None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:43.435652 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-86bfba8d-04b7-493b-bcd4-4d38ff41ddfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:43.507682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 583/1179] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:43.535438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43427b6b-c957-46b8-98d3-3c1d3a302aea None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:43.646559 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43427b6b-c957-46b8-98d3-3c1d3a302aea None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:43.653564 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:43.683362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee36d2-b3f0-4faa-b923-8e629a4e5a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:43.684153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee36d2-b3f0-4faa-b923-8e629a4e5a10 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:43.684441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee36d2-b3f0-4faa-b923-8e629a4e5a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:43.684699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee36d2-b3f0-4faa-b923-8e629a4e5a10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:43.696477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee36d2-b3f0-4faa-b923-8e629a4e5a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d0ee36d2-b3f0-4faa-b923-8e629a4e5a10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:57:43.704603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:57:43.705706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:57:43.707352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:57:43.707352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:57:43.707352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:57:43.707352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:57:43.707352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:57:43.707352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:57:43.707352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:57:43.707352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:57:43.707352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:57:43.707352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:57:43.707352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:43.707352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:57:43.707352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 597/1180] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:57:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:57:43.715949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfba8d-04b7-493b-bcd4-4d38ff41ddfa None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:43.783639 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:43.832474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfba8d-04b7-493b-bcd4-4d38ff41ddfa None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:43.869401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f3385-bd21-4786-bcfc-fed924948184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:43.870093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f3385-bd21-4786-bcfc-fed924948184 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:43.870377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f3385-bd21-4786-bcfc-fed924948184 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:43.870650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f3385-bd21-4786-bcfc-fed924948184 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:43.871394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f3385-bd21-4786-bcfc-fed924948184 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:43.871703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f3385-bd21-4786-bcfc-fed924948184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:43.873095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f3385-bd21-4786-bcfc-fed924948184 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:43.881349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 584/1181] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:57:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:43.887387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee36d2-b3f0-4faa-b923-8e629a4e5a10 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:43.995490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee36d2-b3f0-4faa-b923-8e629a4e5a10 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:44.031670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406891f-56b6-451d-a43d-41a64973faff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:44.032643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406891f-56b6-451d-a43d-41a64973faff None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:44.032919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406891f-56b6-451d-a43d-41a64973faff None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:44.033190 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406891f-56b6-451d-a43d-41a64973faff None admin] PATH_INFO: `/v3/users/7c4d442d6f484aa997619277420ae1ac` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:44.055166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406891f-56b6-451d-a43d-41a64973faff None admin] RBAC: Authorizing `identity:get_user(user_id=7c4d442d6f484aa997619277420ae1ac)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:44.057309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406891f-56b6-451d-a43d-41a64973faff None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:44.059857 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 598/1182] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:43 2022] GET /identity/v3/users/7c4d442d6f484aa997619277420ae1ac => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:44.067149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f3385-bd21-4786-bcfc-fed924948184 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:44.186051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f3385-bd21-4786-bcfc-fed924948184 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:44.202127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:44.221242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f95da-cd4e-462e-826c-e38b38805400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:44.221981 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f95da-cd4e-462e-826c-e38b38805400 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:44.222265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f95da-cd4e-462e-826c-e38b38805400 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:44.222533 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f95da-cd4e-462e-826c-e38b38805400 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:44.290616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f95da-cd4e-462e-826c-e38b38805400 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:44.292071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f95da-cd4e-462e-826c-e38b38805400 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:44.396065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 585/1183] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:57:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:57:44.402604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406891f-56b6-451d-a43d-41a64973faff None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:44.434648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:44.512766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406891f-56b6-451d-a43d-41a64973faff None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:44.548200 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908af37-5850-455e-947b-f0e5ad7d2116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:44.549003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908af37-5850-455e-947b-f0e5ad7d2116 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:44.549285 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908af37-5850-455e-947b-f0e5ad7d2116 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:44.549558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908af37-5850-455e-947b-f0e5ad7d2116 None admin] PATH_INFO: `/v3/projects/f2e68221ac354e17abf3649e73bff0f7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:44.550812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908af37-5850-455e-947b-f0e5ad7d2116 None admin] RBAC: Authorizing `identity:get_project(project_id=f2e68221ac354e17abf3649e73bff0f7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:44.552249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908af37-5850-455e-947b-f0e5ad7d2116 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:44.553977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 599/1184] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:57:44 2022] GET /identity/v3/projects/f2e68221ac354e17abf3649e73bff0f7 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:44.561106 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f95da-cd4e-462e-826c-e38b38805400 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:44.584148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:44.669807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f95da-cd4e-462e-826c-e38b38805400 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:44.704208 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accfcab-65aa-459f-b056-82c9502f44d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:44.704845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accfcab-65aa-459f-b056-82c9502f44d3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:44.705122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accfcab-65aa-459f-b056-82c9502f44d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:44.705396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accfcab-65aa-459f-b056-82c9502f44d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:44.706246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2accfcab-65aa-459f-b056-82c9502f44d3 None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=f2e68221ac354e17abf3649e73bff0f7, user.id=7c4d442d6f484aa997619277420ae1ac) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:44.706558 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2accfcab-65aa-459f-b056-82c9502f44d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:44.707982 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2accfcab-65aa-459f-b056-82c9502f44d3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:44.713462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 586/1185] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:57:44 2022] GET /identity/v3/role_assignments?user.id=7c4d442d6f484aa997619277420ae1ac&scope.project.id=f2e68221ac354e17abf3649e73bff0f7&role.id=a3de9df253294d069dc1867fa26600c3 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:46.716594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d2990-7907-4317-bc91-7122d3dcf028 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:46.717611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d2990-7907-4317-bc91-7122d3dcf028 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:46.717611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d2990-7907-4317-bc91-7122d3dcf028 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:46.719772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 600/1186] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:46.726604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d9f6-4cfc-4e15-9b71-b635f858c5ce None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:46.727062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d9f6-4cfc-4e15-9b71-b635f858c5ce None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:46.727168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d9f6-4cfc-4e15-9b71-b635f858c5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:46.747467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e212d9f6-4cfc-4e15-9b71-b635f858c5ce None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:46.748665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e212d9f6-4cfc-4e15-9b71-b635f858c5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:46.828395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 587/1187] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:46.852661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d2990-7907-4317-bc91-7122d3dcf028 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:46.959480 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:46.974738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d2990-7907-4317-bc91-7122d3dcf028 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:47.011363 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcf073-ff84-48a0-b247-1a848674d15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:47.012150 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcf073-ff84-48a0-b247-1a848674d15f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:47.012432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcf073-ff84-48a0-b247-1a848674d15f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:47.012701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcf073-ff84-48a0-b247-1a848674d15f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:47.023568 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcf073-ff84-48a0-b247-1a848674d15f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-8ebcf073-ff84-48a0-b247-1a848674d15f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:57:47.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:57:47.033860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:57:47.034810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:57:47.034810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:57:47.034810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:57:47.034810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:57:47.034810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:57:47.034810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:57:47.034810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:57:47.034810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:57:47.034810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:57:47.034810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:57:47.034810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:47.034810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:57:47.034810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 601/1188] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:57:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:57:47.040820 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212d9f6-4cfc-4e15-9b71-b635f858c5ce None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:47.058880 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:47.159410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212d9f6-4cfc-4e15-9b71-b635f858c5ce None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:47.195534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17b51d-ad0f-4dc3-9eae-caba3a6681f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:47.196334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17b51d-ad0f-4dc3-9eae-caba3a6681f4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:47.196619 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17b51d-ad0f-4dc3-9eae-caba3a6681f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:47.196894 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17b51d-ad0f-4dc3-9eae-caba3a6681f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:47.197665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17b51d-ad0f-4dc3-9eae-caba3a6681f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:47.197977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17b51d-ad0f-4dc3-9eae-caba3a6681f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:47.199380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17b51d-ad0f-4dc3-9eae-caba3a6681f4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:47.208598 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 588/1189] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:57:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:47.216333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcf073-ff84-48a0-b247-1a848674d15f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:47.322461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcf073-ff84-48a0-b247-1a848674d15f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:47.359536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfb0c9-7122-432f-bd3e-92ddf80695e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:47.360718 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfb0c9-7122-432f-bd3e-92ddf80695e8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:47.361008 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfb0c9-7122-432f-bd3e-92ddf80695e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:47.361287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfb0c9-7122-432f-bd3e-92ddf80695e8 None admin] PATH_INFO: `/v3/users/7c4d442d6f484aa997619277420ae1ac` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:47.362641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfb0c9-7122-432f-bd3e-92ddf80695e8 None admin] RBAC: Authorizing `identity:get_user(user_id=7c4d442d6f484aa997619277420ae1ac)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:47.364116 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfb0c9-7122-432f-bd3e-92ddf80695e8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:47.365764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 602/1190] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:47 2022] GET /identity/v3/users/7c4d442d6f484aa997619277420ae1ac => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:47.372269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17b51d-ad0f-4dc3-9eae-caba3a6681f4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:47.454277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:47.486349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17b51d-ad0f-4dc3-9eae-caba3a6681f4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:47.522097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11bd2e-1944-44be-9646-15ce31be3ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:47.522960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11bd2e-1944-44be-9646-15ce31be3ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:47.523239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11bd2e-1944-44be-9646-15ce31be3ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:47.523512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11bd2e-1944-44be-9646-15ce31be3ca6 None admin] PATH_INFO: `/v3/projects/f2e68221ac354e17abf3649e73bff0f7` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:47.524805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11bd2e-1944-44be-9646-15ce31be3ca6 None admin] RBAC: Authorizing `identity:get_project(project_id=f2e68221ac354e17abf3649e73bff0f7)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:47.526300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11bd2e-1944-44be-9646-15ce31be3ca6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:47.528002 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 589/1191] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:57:47 2022] GET /identity/v3/projects/f2e68221ac354e17abf3649e73bff0f7 => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:47.534663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfb0c9-7122-432f-bd3e-92ddf80695e8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:47.558971 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:47.641730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfb0c9-7122-432f-bd3e-92ddf80695e8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:47.678233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4dbfd-2afc-46dc-90f0-d6b426cc3387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:47.679111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4dbfd-2afc-46dc-90f0-d6b426cc3387 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:47.679397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4dbfd-2afc-46dc-90f0-d6b426cc3387 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:47.679653 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4dbfd-2afc-46dc-90f0-d6b426cc3387 None admin] PATH_INFO: `/v3/projects/f2e68221ac354e17abf3649e73bff0f7/users/7c4d442d6f484aa997619277420ae1ac/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:47.681646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4dbfd-2afc-46dc-90f0-d6b426cc3387 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2e68221ac354e17abf3649e73bff0f7, user_id=7c4d442d6f484aa997619277420ae1ac, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:47.683119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4dbfd-2afc-46dc-90f0-d6b426cc3387 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:47.698589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 603/1192] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:57:47 2022] PUT /identity/v3/projects/f2e68221ac354e17abf3649e73bff0f7/users/7c4d442d6f484aa997619277420ae1ac/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:57:49.658132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7aafa-bd5d-4748-adb1-9fdf7bf1d913 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:49.658720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7aafa-bd5d-4748-adb1-9fdf7bf1d913 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:49.658720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7aafa-bd5d-4748-adb1-9fdf7bf1d913 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:49.660073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 590/1193] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:57:49.666405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dcb8f-e6ee-45bd-be38-badc5b7bc352 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:49.667122 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dcb8f-e6ee-45bd-be38-badc5b7bc352 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:49.667278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dcb8f-e6ee-45bd-be38-badc5b7bc352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:49.686544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-cb9dcb8f-e6ee-45bd-be38-badc5b7bc352 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:57:49.687736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-cb9dcb8f-e6ee-45bd-be38-badc5b7bc352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:57:49.759552 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 604/1194] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:57:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:57:49.783540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7aafa-bd5d-4748-adb1-9fdf7bf1d913 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:49.891930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7aafa-bd5d-4748-adb1-9fdf7bf1d913 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:49.926749 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef58bb-f678-43b6-9251-9f8ca4790f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:49.927463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef58bb-f678-43b6-9251-9f8ca4790f23 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:49.927740 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef58bb-f678-43b6-9251-9f8ca4790f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:49.928011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef58bb-f678-43b6-9251-9f8ca4790f23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:49.935794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:49.939595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef58bb-f678-43b6-9251-9f8ca4790f23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-38ef58bb-f678-43b6-9251-9f8ca4790f23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:57:49.947498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:57:49.948627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:57:49.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:57:49.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:57:49.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:57:49.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:57:49.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:57:49.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:57:49.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:57:49.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:57:49.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:57:49.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:57:49.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:57:49.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:57:49.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 591/1195] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:57:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:57:49.955473 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dcb8f-e6ee-45bd-be38-badc5b7bc352 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:49.972348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:50.074438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dcb8f-e6ee-45bd-be38-badc5b7bc352 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:50.110672 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6f41a-9697-406a-88e9-0285e1866ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:50.111331 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6f41a-9697-406a-88e9-0285e1866ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:50.111621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6f41a-9697-406a-88e9-0285e1866ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:50.111897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6f41a-9697-406a-88e9-0285e1866ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:50.112642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6f41a-9697-406a-88e9-0285e1866ff6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:50.112949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6f41a-9697-406a-88e9-0285e1866ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:50.114357 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6f41a-9697-406a-88e9-0285e1866ff6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:50.122537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 605/1196] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:57:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:50.129304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef58bb-f678-43b6-9251-9f8ca4790f23 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:50.249595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef58bb-f678-43b6-9251-9f8ca4790f23 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:50.284458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84458586-82dd-4ad2-ac23-6663ef9f141a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:50.285327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84458586-82dd-4ad2-ac23-6663ef9f141a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:50.285605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84458586-82dd-4ad2-ac23-6663ef9f141a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:50.285904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84458586-82dd-4ad2-ac23-6663ef9f141a None admin] PATH_INFO: `/v3/users/7c4d442d6f484aa997619277420ae1ac` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:50.287156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84458586-82dd-4ad2-ac23-6663ef9f141a None admin] RBAC: Authorizing `identity:get_user(user_id=7c4d442d6f484aa997619277420ae1ac)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:50.288623 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84458586-82dd-4ad2-ac23-6663ef9f141a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:50.290300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 592/1197] 10.222.0.79 () {56 vars in 1177 bytes} [Mon Jul 4 09:57:50 2022] GET /identity/v3/users/7c4d442d6f484aa997619277420ae1ac => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:57:50.296617 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6f41a-9697-406a-88e9-0285e1866ff6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:50.370290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:50.402219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6f41a-9697-406a-88e9-0285e1866ff6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:50.438228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f92714-8f55-4b8d-b3bc-b21bd6e7b032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:50.438869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f92714-8f55-4b8d-b3bc-b21bd6e7b032 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:50.439303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f92714-8f55-4b8d-b3bc-b21bd6e7b032 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:50.439582 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f92714-8f55-4b8d-b3bc-b21bd6e7b032 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:50.507941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f92714-8f55-4b8d-b3bc-b21bd6e7b032 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:50.509311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f92714-8f55-4b8d-b3bc-b21bd6e7b032 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:50.581850 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:50.613186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 606/1198] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:57:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:57:50.619391 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84458586-82dd-4ad2-ac23-6663ef9f141a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:50.688051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:57:50.721936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84458586-82dd-4ad2-ac23-6663ef9f141a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:50.757418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5de5d6-d94c-48ce-9ed5-6cd39f51de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:50.758361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5de5d6-d94c-48ce-9ed5-6cd39f51de4c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:50.758583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5de5d6-d94c-48ce-9ed5-6cd39f51de4c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:50.758856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5de5d6-d94c-48ce-9ed5-6cd39f51de4c None admin] PATH_INFO: `/v3/projects/f2e68221ac354e17abf3649e73bff0f7` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:50.760152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5de5d6-d94c-48ce-9ed5-6cd39f51de4c None admin] RBAC: Authorizing `identity:get_project(project_id=f2e68221ac354e17abf3649e73bff0f7)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:50.761602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5de5d6-d94c-48ce-9ed5-6cd39f51de4c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:50.763323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 593/1199] 10.222.0.79 () {56 vars in 1186 bytes} [Mon Jul 4 09:57:50 2022] GET /identity/v3/projects/f2e68221ac354e17abf3649e73bff0f7 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:57:50.770274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f92714-8f55-4b8d-b3bc-b21bd6e7b032 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:57:50.877946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f92714-8f55-4b8d-b3bc-b21bd6e7b032 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:57:50.914805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f0a18-1078-4edb-adcc-4ae893c1d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:57:50.915452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f0a18-1078-4edb-adcc-4ae893c1d5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:57:50.915735 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f0a18-1078-4edb-adcc-4ae893c1d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:57:50.916010 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f0a18-1078-4edb-adcc-4ae893c1d5d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:57:50.916835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f0a18-1078-4edb-adcc-4ae893c1d5d1 None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=f2e68221ac354e17abf3649e73bff0f7, user.id=7c4d442d6f484aa997619277420ae1ac) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:57:50.917149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f0a18-1078-4edb-adcc-4ae893c1d5d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:57:50.918619 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f0a18-1078-4edb-adcc-4ae893c1d5d1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:57:50.925185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 607/1200] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:57:50 2022] GET /identity/v3/role_assignments?user.id=7c4d442d6f484aa997619277420ae1ac&scope.project.id=f2e68221ac354e17abf3649e73bff0f7&role.id=a3de9df253294d069dc1867fa26600c3 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:00.753248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=32 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:00.921839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=82 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:46.731879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bf8f1-6ece-43d2-9c55-ec34447c4f66 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:46.732772 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bf8f1-6ece-43d2-9c55-ec34447c4f66 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:46.732772 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bf8f1-6ece-43d2-9c55-ec34447c4f66 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:46.734238 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 594/1201] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:58:46.741140 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9c242-c7cb-47ed-8969-53fcdf151d9a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:46.741786 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9c242-c7cb-47ed-8969-53fcdf151d9a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:46.742414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9c242-c7cb-47ed-8969-53fcdf151d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:46.776185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3ca9c242-c7cb-47ed-8969-53fcdf151d9a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:58:46.778581 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3ca9c242-c7cb-47ed-8969-53fcdf151d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:58:46.883945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 608/1202] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:58:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:58:46.910572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bf8f1-6ece-43d2-9c55-ec34447c4f66 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:47.050542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bf8f1-6ece-43d2-9c55-ec34447c4f66 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:47.102348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf02b8-7486-4255-aa79-814e44a57ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:47.103556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf02b8-7486-4255-aa79-814e44a57ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:47.103695 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf02b8-7486-4255-aa79-814e44a57ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:47.104195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf02b8-7486-4255-aa79-814e44a57ef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:47.116597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf02b8-7486-4255-aa79-814e44a57ef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:47.118140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf02b8-7486-4255-aa79-814e44a57ef6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-9faf02b8-7486-4255-aa79-814e44a57ef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:58:47.147157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:47.148231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:47.150071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:58:47.150071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:58:47.150071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:58:47.150071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:58:47.150071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 595/1203] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:58:47.155253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9c242-c7cb-47ed-8969-53fcdf151d9a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:47.299994 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9c242-c7cb-47ed-8969-53fcdf151d9a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:47.336441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:47.343192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e1340-96c2-429b-bdd7-96761fefa403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:47.344086 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e1340-96c2-429b-bdd7-96761fefa403 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:47.344528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e1340-96c2-429b-bdd7-96761fefa403 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:47.345027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e1340-96c2-429b-bdd7-96761fefa403 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:47.345792 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e1340-96c2-429b-bdd7-96761fefa403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:47.346140 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e1340-96c2-429b-bdd7-96761fefa403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:47.347692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e1340-96c2-429b-bdd7-96761fefa403 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:47.387683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 609/1204] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:47.388953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf02b8-7486-4255-aa79-814e44a57ef6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:47.463971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:47.540647 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf02b8-7486-4255-aa79-814e44a57ef6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:47.599386 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c7526-baf2-4f86-997d-c2be13890fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:47.600444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c7526-baf2-4f86-997d-c2be13890fa9 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:47.600954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c7526-baf2-4f86-997d-c2be13890fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:47.601435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c7526-baf2-4f86-997d-c2be13890fa9 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:47.602755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c7526-baf2-4f86-997d-c2be13890fa9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:47.605280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c7526-baf2-4f86-997d-c2be13890fa9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:47.641891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 596/1205] 10.222.0.79 () {60 vars in 1132 bytes} [Mon Jul 4 09:58:47 2022] POST /identity/v3/users => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:58:49.882386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df5ee4-a5f2-4c15-a9f2-a3f65096f0cd None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:49.883175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df5ee4-a5f2-4c15-a9f2-a3f65096f0cd None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:49.883175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df5ee4-a5f2-4c15-a9f2-a3f65096f0cd None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:49.884858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 610/1206] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:58:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:58:49.890229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a3bc-0aff-487c-9466-47d3436dd1f0 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:49.891248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a3bc-0aff-487c-9466-47d3436dd1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:49.891539 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a3bc-0aff-487c-9466-47d3436dd1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:49.909211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4635a3bc-0aff-487c-9466-47d3436dd1f0 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:58:49.911024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4635a3bc-0aff-487c-9466-47d3436dd1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:58:49.987898 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 597/1207] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:58:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:58:50.012256 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df5ee4-a5f2-4c15-a9f2-a3f65096f0cd None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:50.138374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df5ee4-a5f2-4c15-a9f2-a3f65096f0cd None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:50.178923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97873e-0e82-45ea-b574-2b12f2fbf394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:50.179910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97873e-0e82-45ea-b574-2b12f2fbf394 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:50.180168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97873e-0e82-45ea-b574-2b12f2fbf394 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:50.180443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97873e-0e82-45ea-b574-2b12f2fbf394 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:50.192763 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97873e-0e82-45ea-b574-2b12f2fbf394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-ca97873e-0e82-45ea-b574-2b12f2fbf394 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:58:50.200746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:50.202052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:50.203720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:50.203720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:50.203720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:50.203720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:50.203720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:58:50.203720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:58:50.203720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:58:50.203720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:58:50.203720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:58:50.203720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:58:50.203720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:58:50.203720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:58:50.203720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 611/1208] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:58:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:58:50.208442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635a3bc-0aff-487c-9466-47d3436dd1f0 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:50.251672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:50.321252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635a3bc-0aff-487c-9466-47d3436dd1f0 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:50.364071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb74e2-d573-4ee6-98d2-de70fbe811db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:50.364841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb74e2-d573-4ee6-98d2-de70fbe811db None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:50.365152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb74e2-d573-4ee6-98d2-de70fbe811db None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:50.365462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb74e2-d573-4ee6-98d2-de70fbe811db None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:50.366373 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb74e2-d573-4ee6-98d2-de70fbe811db None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:50.366720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb74e2-d573-4ee6-98d2-de70fbe811db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:50.368248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb74e2-d573-4ee6-98d2-de70fbe811db None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:50.379175 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 598/1209] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:58:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:58:50.385578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97873e-0e82-45ea-b574-2b12f2fbf394 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:50.448305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:50.509812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97873e-0e82-45ea-b574-2b12f2fbf394 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:50.552962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac992eb-1f3e-40e1-b84c-d7aff6bcd235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:50.552962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac992eb-1f3e-40e1-b84c-d7aff6bcd235 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:50.552962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac992eb-1f3e-40e1-b84c-d7aff6bcd235 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:50.552962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac992eb-1f3e-40e1-b84c-d7aff6bcd235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:50.564595 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac992eb-1f3e-40e1-b84c-d7aff6bcd235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:50.566295 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac992eb-1f3e-40e1-b84c-d7aff6bcd235 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-dac992eb-1f3e-40e1-b84c-d7aff6bcd235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:58:50.586846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:50.587963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:50.588863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:58:50.588863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:58:50.588863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:58:50.588863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:58:50.588863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 612/1210] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:58:50.594663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb74e2-d573-4ee6-98d2-de70fbe811db None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:50.702326 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:50.705538 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb74e2-d573-4ee6-98d2-de70fbe811db None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:50.740635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2c16b-428e-4a18-b7f7-8bfa8abbbe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:50.741281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2c16b-428e-4a18-b7f7-8bfa8abbbe07 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:50.741560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2c16b-428e-4a18-b7f7-8bfa8abbbe07 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:50.741864 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2c16b-428e-4a18-b7f7-8bfa8abbbe07 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:50.742586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2c16b-428e-4a18-b7f7-8bfa8abbbe07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:50.742896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2c16b-428e-4a18-b7f7-8bfa8abbbe07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:50.744288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2c16b-428e-4a18-b7f7-8bfa8abbbe07 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:50.761473 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 599/1211] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:50.762848 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac992eb-1f3e-40e1-b84c-d7aff6bcd235 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:50.876575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac992eb-1f3e-40e1-b84c-d7aff6bcd235 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:50.878650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:50.924151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ac5bf-e26c-4542-b8d8-b7241154b926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:50.924988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ac5bf-e26c-4542-b8d8-b7241154b926 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:50.925249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ac5bf-e26c-4542-b8d8-b7241154b926 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:50.927139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ac5bf-e26c-4542-b8d8-b7241154b926 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:50.935303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ac5bf-e26c-4542-b8d8-b7241154b926 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:50.936792 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ac5bf-e26c-4542-b8d8-b7241154b926 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b63ac5bf-e26c-4542-b8d8-b7241154b926 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:58:50.954515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:50.955560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:50.956525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:50.956525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:50.956525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:58:50.956525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:58:50.956525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:58:50.956525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:58:50.956525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:58:50.956525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:58:50.956525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:58:50.956525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:58:50.956525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 613/1212] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:58:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:58:50.961889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2c16b-428e-4a18-b7f7-8bfa8abbbe07 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:51.082235 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2c16b-428e-4a18-b7f7-8bfa8abbbe07 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:51.123255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a1636-90c1-4692-8552-15bebea19292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:51.123957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a1636-90c1-4692-8552-15bebea19292 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:51.124240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a1636-90c1-4692-8552-15bebea19292 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:51.124676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a1636-90c1-4692-8552-15bebea19292 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:51.127903 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a1636-90c1-4692-8552-15bebea19292 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:51.127903 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a1636-90c1-4692-8552-15bebea19292 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:51.127903 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a1636-90c1-4692-8552-15bebea19292 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:51.150010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 600/1213] 10.222.0.79 () {56 vars in 1143 bytes} [Mon Jul 4 09:58:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:58:51.156458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ac5bf-e26c-4542-b8d8-b7241154b926 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:51.269531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ac5bf-e26c-4542-b8d8-b7241154b926 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:51.309287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227b03c-eaac-450c-bddb-7af7902426d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:51.311021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227b03c-eaac-450c-bddb-7af7902426d4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:51.311021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227b03c-eaac-450c-bddb-7af7902426d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:51.311021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227b03c-eaac-450c-bddb-7af7902426d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:51.344234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:51.387072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227b03c-eaac-450c-bddb-7af7902426d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:51.388682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227b03c-eaac-450c-bddb-7af7902426d4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:51.501878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 614/1214] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:58:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:58:51.508104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a1636-90c1-4692-8552-15bebea19292 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:51.530858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:51.621176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a1636-90c1-4692-8552-15bebea19292 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:51.658202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe20b9f-9422-4212-a679-f338f9132500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:51.659099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe20b9f-9422-4212-a679-f338f9132500 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:51.659406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe20b9f-9422-4212-a679-f338f9132500 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:51.659667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe20b9f-9422-4212-a679-f338f9132500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:51.671273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe20b9f-9422-4212-a679-f338f9132500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:51.672756 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe20b9f-9422-4212-a679-f338f9132500 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-7fe20b9f-9422-4212-a679-f338f9132500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:58:51.692728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:51.693916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:51.695303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:58:51.695303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:58:51.695303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:58:51.695303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:58:51.695303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 601/1215] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:58:51.700757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227b03c-eaac-450c-bddb-7af7902426d4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:51.778077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:51.819039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227b03c-eaac-450c-bddb-7af7902426d4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:51.857898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da52c3-83ed-4730-b38c-4f39c838cd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:51.858578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da52c3-83ed-4730-b38c-4f39c838cd72 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:51.859108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da52c3-83ed-4730-b38c-4f39c838cd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:51.859334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da52c3-83ed-4730-b38c-4f39c838cd72 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:51.860064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da52c3-83ed-4730-b38c-4f39c838cd72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:51.860379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da52c3-83ed-4730-b38c-4f39c838cd72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:51.861850 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da52c3-83ed-4730-b38c-4f39c838cd72 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:51.875969 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 615/1216] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:51.882665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe20b9f-9422-4212-a679-f338f9132500 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:51.969006 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:52.006783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe20b9f-9422-4212-a679-f338f9132500 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:52.046226 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0958980-aed0-485e-a5e0-e9869da5c082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:52.047367 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0958980-aed0-485e-a5e0-e9869da5c082 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:52.047660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0958980-aed0-485e-a5e0-e9869da5c082 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:52.047946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0958980-aed0-485e-a5e0-e9869da5c082 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:52.054995 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0958980-aed0-485e-a5e0-e9869da5c082 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:52.056471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0958980-aed0-485e-a5e0-e9869da5c082 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d0958980-aed0-485e-a5e0-e9869da5c082 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:58:52.070875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:52.071867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:52.072805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:58:52.072805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:58:52.072805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:58:52.072805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:58:52.072805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:58:52.072805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:58:52.072805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:58:52.072805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:58:52.072805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 602/1217] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:58:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:58:52.081671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da52c3-83ed-4730-b38c-4f39c838cd72 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:52.194054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da52c3-83ed-4730-b38c-4f39c838cd72 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:52.219441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:52.233312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508bf76-328a-49ce-a0be-671661fb23b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:52.234644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508bf76-328a-49ce-a0be-671661fb23b5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:52.235101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508bf76-328a-49ce-a0be-671661fb23b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:52.236474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508bf76-328a-49ce-a0be-671661fb23b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:52.236474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508bf76-328a-49ce-a0be-671661fb23b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:52.236777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508bf76-328a-49ce-a0be-671661fb23b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:52.238317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508bf76-328a-49ce-a0be-671661fb23b5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:52.256199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 616/1218] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:58:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:52.258022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0958980-aed0-485e-a5e0-e9869da5c082 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:52.369128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0958980-aed0-485e-a5e0-e9869da5c082 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:52.402661 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:52.409419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38319f-89d7-4a8d-b11b-16a5996f0a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:52.410560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38319f-89d7-4a8d-b11b-16a5996f0a67 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:52.410824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38319f-89d7-4a8d-b11b-16a5996f0a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:52.411269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38319f-89d7-4a8d-b11b-16a5996f0a67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:52.412263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38319f-89d7-4a8d-b11b-16a5996f0a67 None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=5dc48318ab7944c197bf5f7673511931) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:52.412608 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38319f-89d7-4a8d-b11b-16a5996f0a67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:52.414214 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38319f-89d7-4a8d-b11b-16a5996f0a67 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:52.420519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 603/1219] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:58:52 2022] GET /identity/v3/role_assignments?user.id=5dc48318ab7944c197bf5f7673511931&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:54.445651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3176-eca6-4cc5-bd46-462600438ed8 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:54.446182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3176-eca6-4cc5-bd46-462600438ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:54.446182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3176-eca6-4cc5-bd46-462600438ed8 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:54.447613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 617/1220] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:58:54.453870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a718b93-cc8b-4a1a-9f43-6d45caa7e796 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:54.454360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a718b93-cc8b-4a1a-9f43-6d45caa7e796 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:54.454565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a718b93-cc8b-4a1a-9f43-6d45caa7e796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:54.475039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2a718b93-cc8b-4a1a-9f43-6d45caa7e796 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:58:54.476647 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2a718b93-cc8b-4a1a-9f43-6d45caa7e796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:58:54.553043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 604/1221] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:58:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:58:54.575781 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d3176-eca6-4cc5-bd46-462600438ed8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:54.697401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d3176-eca6-4cc5-bd46-462600438ed8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:54.739161 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaa1c8-9e25-43a4-9475-d0d3d48f0e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:54.739946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaa1c8-9e25-43a4-9475-d0d3d48f0e5e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:54.740252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaa1c8-9e25-43a4-9475-d0d3d48f0e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:54.740511 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaa1c8-9e25-43a4-9475-d0d3d48f0e5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:54.754664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaa1c8-9e25-43a4-9475-d0d3d48f0e5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-feeaa1c8-9e25-43a4-9475-d0d3d48f0e5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:58:54.762443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:54.763546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:54.764542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:54.764542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:54.764542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:54.764542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:54.764542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:58:54.764542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:58:54.764542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:58:54.764542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:58:54.764542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:58:54.764542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:58:54.764542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:58:54.764542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:58:54.764542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 618/1222] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:58:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:58:54.770284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a718b93-cc8b-4a1a-9f43-6d45caa7e796 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:54.881058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a718b93-cc8b-4a1a-9f43-6d45caa7e796 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:54.905318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:54.917231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357aacf-cf84-4767-a139-c0dffa466d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:54.917934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357aacf-cf84-4767-a139-c0dffa466d00 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:54.918219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357aacf-cf84-4767-a139-c0dffa466d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:54.918543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357aacf-cf84-4767-a139-c0dffa466d00 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:54.919300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357aacf-cf84-4767-a139-c0dffa466d00 None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:54.919607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357aacf-cf84-4767-a139-c0dffa466d00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:54.921012 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357aacf-cf84-4767-a139-c0dffa466d00 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:54.929310 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 605/1223] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:58:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:54.936447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaa1c8-9e25-43a4-9475-d0d3d48f0e5e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:54.981970 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:55.044194 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaa1c8-9e25-43a4-9475-d0d3d48f0e5e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:55.080455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894080c8-1ade-43c0-8130-45c1795fce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:55.081438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894080c8-1ade-43c0-8130-45c1795fce9c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:55.081749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894080c8-1ade-43c0-8130-45c1795fce9c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:55.082048 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894080c8-1ade-43c0-8130-45c1795fce9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:55.092542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894080c8-1ade-43c0-8130-45c1795fce9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:55.094460 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894080c8-1ade-43c0-8130-45c1795fce9c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-894080c8-1ade-43c0-8130-45c1795fce9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:58:55.112576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:55.113742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:55.114670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:58:55.114670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:58:55.114670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:58:55.114670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:58:55.114670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 619/1224] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:58:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:58:55.119949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357aacf-cf84-4767-a139-c0dffa466d00 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:55.234130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357aacf-cf84-4767-a139-c0dffa466d00 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:55.269929 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ba360-db1b-4e9b-9568-8963d6cad35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:55.270677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ba360-db1b-4e9b-9568-8963d6cad35f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:55.270920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ba360-db1b-4e9b-9568-8963d6cad35f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:55.271232 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ba360-db1b-4e9b-9568-8963d6cad35f None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:55.271956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ba360-db1b-4e9b-9568-8963d6cad35f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:55.272236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ba360-db1b-4e9b-9568-8963d6cad35f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:55.273629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ba360-db1b-4e9b-9568-8963d6cad35f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:55.286504 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 606/1225] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:55.292251 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894080c8-1ade-43c0-8130-45c1795fce9c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:55.392672 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:55.400550 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894080c8-1ade-43c0-8130-45c1795fce9c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:55.433566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fccc4-e3a6-445e-85b9-102235125b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:55.434388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fccc4-e3a6-445e-85b9-102235125b10 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:55.434679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fccc4-e3a6-445e-85b9-102235125b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:55.434952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fccc4-e3a6-445e-85b9-102235125b10 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:55.442874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740fccc4-e3a6-445e-85b9-102235125b10 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:55.444324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740fccc4-e3a6-445e-85b9-102235125b10 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-740fccc4-e3a6-445e-85b9-102235125b10 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:58:55.458596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:55.465501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:55.466502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:55.466502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:55.466502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:58:55.466502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:58:55.466502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:58:55.466502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:58:55.466502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:58:55.466502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:58:55.466502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:58:55.466502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:58:55.466502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 620/1226] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:58:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:58:55.467095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ba360-db1b-4e9b-9568-8963d6cad35f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:55.514256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:55.567664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ba360-db1b-4e9b-9568-8963d6cad35f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:55.605846 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7dd1b5-18df-4793-87e3-5fbfa8415d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:55.606605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dd1b5-18df-4793-87e3-5fbfa8415d12 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:55.607000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dd1b5-18df-4793-87e3-5fbfa8415d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:55.607318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dd1b5-18df-4793-87e3-5fbfa8415d12 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:55.608145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7dd1b5-18df-4793-87e3-5fbfa8415d12 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:55.608454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7dd1b5-18df-4793-87e3-5fbfa8415d12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:55.609987 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7dd1b5-18df-4793-87e3-5fbfa8415d12 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:55.627930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 607/1227] 10.222.0.79 () {56 vars in 1143 bytes} [Mon Jul 4 09:58:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:55.634056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fccc4-e3a6-445e-85b9-102235125b10 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:55.743749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fccc4-e3a6-445e-85b9-102235125b10 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:55.780897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70a77c-9fc3-4eb5-a907-c06d44703fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:55.781734 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70a77c-9fc3-4eb5-a907-c06d44703fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:55.782021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70a77c-9fc3-4eb5-a907-c06d44703fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:55.782301 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70a77c-9fc3-4eb5-a907-c06d44703fc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:55.793230 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70a77c-9fc3-4eb5-a907-c06d44703fc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:55.794752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70a77c-9fc3-4eb5-a907-c06d44703fc6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-ca70a77c-9fc3-4eb5-a907-c06d44703fc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:58:55.811809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:55.813171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:55.814224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:58:55.814224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:58:55.814224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:58:55.814224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:58:55.814224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 621/1228] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:58:55.819520 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7dd1b5-18df-4793-87e3-5fbfa8415d12 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:55.935032 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7dd1b5-18df-4793-87e3-5fbfa8415d12 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:55.937578 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:55.974478 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db819141-0a6c-43af-827a-866c4a465895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:55.975310 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db819141-0a6c-43af-827a-866c4a465895 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:55.975596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db819141-0a6c-43af-827a-866c4a465895 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:55.975880 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db819141-0a6c-43af-827a-866c4a465895 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:55.976708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db819141-0a6c-43af-827a-866c4a465895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:55.977069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db819141-0a6c-43af-827a-866c4a465895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:55.978642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db819141-0a6c-43af-827a-866c4a465895 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:55.991337 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 608/1229] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:55.997182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70a77c-9fc3-4eb5-a907-c06d44703fc6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:56.006962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:56.106398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70a77c-9fc3-4eb5-a907-c06d44703fc6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:56.143447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aee542-ed0e-428f-9b86-79a735fab26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:56.144560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee542-ed0e-428f-9b86-79a735fab26e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:56.144849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee542-ed0e-428f-9b86-79a735fab26e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:56.145165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee542-ed0e-428f-9b86-79a735fab26e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:56.152510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aee542-ed0e-428f-9b86-79a735fab26e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:56.154139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aee542-ed0e-428f-9b86-79a735fab26e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-41aee542-ed0e-428f-9b86-79a735fab26e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:58:56.169471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:56.170448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:56.172108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:58:56.172108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:58:56.172108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:58:56.172108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:58:56.172108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:58:56.172108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:58:56.172108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:58:56.172108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:58:56.172108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 622/1230] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:58:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:58:56.176835 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db819141-0a6c-43af-827a-866c4a465895 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:56.293395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db819141-0a6c-43af-827a-866c4a465895 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:56.332495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e1405-0f2f-4bf4-900f-ef74778a0545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:56.333204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e1405-0f2f-4bf4-900f-ef74778a0545 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:56.333492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e1405-0f2f-4bf4-900f-ef74778a0545 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:56.333922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e1405-0f2f-4bf4-900f-ef74778a0545 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:56.334674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e1405-0f2f-4bf4-900f-ef74778a0545 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:56.335060 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e1405-0f2f-4bf4-900f-ef74778a0545 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:56.336579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e1405-0f2f-4bf4-900f-ef74778a0545 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:56.351252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 609/1231] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:58:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:56.357476 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aee542-ed0e-428f-9b86-79a735fab26e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:56.433593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:56.469263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aee542-ed0e-428f-9b86-79a735fab26e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:56.506337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7adaa-5e23-4430-885d-b772c1a40511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:56.507235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7adaa-5e23-4430-885d-b772c1a40511 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:56.507515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7adaa-5e23-4430-885d-b772c1a40511 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:56.507788 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7adaa-5e23-4430-885d-b772c1a40511 None admin] PATH_INFO: `/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/5dc48318ab7944c197bf5f7673511931/roles/fc6661dcb3994755a5c238bf37e17bb9` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:56.523154 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7adaa-5e23-4430-885d-b772c1a40511 None admin] RBAC: Authorizing `identity:create_grant(project_id=f32306262e8744eab4bdc6d28e24e6d0, user_id=5dc48318ab7944c197bf5f7673511931, role_id=fc6661dcb3994755a5c238bf37e17bb9)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:56.524981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7adaa-5e23-4430-885d-b772c1a40511 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:56.540285 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 623/1232] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:58:56 2022] PUT /identity/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/5dc48318ab7944c197bf5f7673511931/roles/fc6661dcb3994755a5c238bf37e17bb9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:58:58.674095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b79b4f-429b-4465-9bfe-026be5c86422 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:58.674675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b79b4f-429b-4465-9bfe-026be5c86422 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:58.674675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b79b4f-429b-4465-9bfe-026be5c86422 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:58.676042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 610/1233] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:58:58.681294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0a470-bf46-4535-b627-48a81c726e86 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:58.681748 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0a470-bf46-4535-b627-48a81c726e86 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:58.681996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0a470-bf46-4535-b627-48a81c726e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:58.700274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-90a0a470-bf46-4535-b627-48a81c726e86 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:58:58.701574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-90a0a470-bf46-4535-b627-48a81c726e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:58:58.779646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 624/1234] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:58:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:58:58.803608 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b79b4f-429b-4465-9bfe-026be5c86422 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:58.833278 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:58.920910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b79b4f-429b-4465-9bfe-026be5c86422 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:58.959721 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d1d82-7a7f-4839-94c3-518d7e6f128d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:58.960428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d1d82-7a7f-4839-94c3-518d7e6f128d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:58.960706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d1d82-7a7f-4839-94c3-518d7e6f128d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:58.960977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d1d82-7a7f-4839-94c3-518d7e6f128d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:58.970966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d1d82-7a7f-4839-94c3-518d7e6f128d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-3b7d1d82-7a7f-4839-94c3-518d7e6f128d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:58:58.979666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:58.980852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:58.981858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:58.981858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:58.981858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:58.981858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:58.981858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:58:58.981858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:58:58.981858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:58:58.981858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:58:58.981858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:58:58.981858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:58:58.981858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:58:58.981858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:58:58.981858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 611/1235] 10.222.0.79 () {56 vars in 1102 bytes} [Mon Jul 4 09:58:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:58:58.988925 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0a470-bf46-4535-b627-48a81c726e86 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:59.035840 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:59.093098 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0a470-bf46-4535-b627-48a81c726e86 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:59.128294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8795a56-cf9c-418e-b72b-bdcef0f9ff3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:59.128973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8795a56-cf9c-418e-b72b-bdcef0f9ff3a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:59.129257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8795a56-cf9c-418e-b72b-bdcef0f9ff3a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:59.129631 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8795a56-cf9c-418e-b72b-bdcef0f9ff3a None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:59.130593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8795a56-cf9c-418e-b72b-bdcef0f9ff3a None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:59.130941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8795a56-cf9c-418e-b72b-bdcef0f9ff3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:59.132448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8795a56-cf9c-418e-b72b-bdcef0f9ff3a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:59.143744 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 625/1236] 10.222.0.79 () {56 vars in 1103 bytes} [Mon Jul 4 09:58:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:59.149786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d1d82-7a7f-4839-94c3-518d7e6f128d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:59.246035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:59.263337 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d1d82-7a7f-4839-94c3-518d7e6f128d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:59.299734 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6eda1-e851-42cb-8fe9-83adfb42f21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:59.300599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6eda1-e851-42cb-8fe9-83adfb42f21f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:59.300922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6eda1-e851-42cb-8fe9-83adfb42f21f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:59.301228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6eda1-e851-42cb-8fe9-83adfb42f21f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:59.313241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6eda1-e851-42cb-8fe9-83adfb42f21f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:59.314966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6eda1-e851-42cb-8fe9-83adfb42f21f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1be6eda1-e851-42cb-8fe9-83adfb42f21f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:58:59.333227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:59.334410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:59.335414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:58:59.335414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:58:59.335414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:58:59.335414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:58:59.335414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 612/1237] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:58:59.340571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8795a56-cf9c-418e-b72b-bdcef0f9ff3a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:59.448676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8795a56-cf9c-418e-b72b-bdcef0f9ff3a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:59.459516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:59.488287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0293b1c-0863-4bf1-8556-22ced3ead5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:59.488973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0293b1c-0863-4bf1-8556-22ced3ead5bd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:59.489253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0293b1c-0863-4bf1-8556-22ced3ead5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:59.489526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0293b1c-0863-4bf1-8556-22ced3ead5bd None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:59.490299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0293b1c-0863-4bf1-8556-22ced3ead5bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:59.490604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0293b1c-0863-4bf1-8556-22ced3ead5bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:59.492046 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0293b1c-0863-4bf1-8556-22ced3ead5bd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:59.504121 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 626/1238] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:59.509896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6eda1-e851-42cb-8fe9-83adfb42f21f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:59.620513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6eda1-e851-42cb-8fe9-83adfb42f21f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:59.655019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07d592-0786-4d80-b885-bcbe8d83c33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:59.655960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07d592-0786-4d80-b885-bcbe8d83c33b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:59.656273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07d592-0786-4d80-b885-bcbe8d83c33b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:59.656546 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07d592-0786-4d80-b885-bcbe8d83c33b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:59.659799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:58:59.665664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07d592-0786-4d80-b885-bcbe8d83c33b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:59.667268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07d592-0786-4d80-b885-bcbe8d83c33b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-6b07d592-0786-4d80-b885-bcbe8d83c33b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:58:59.682438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:58:59.683962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:58:59.685178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:58:59.685178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:58:59.685178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:58:59.685178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:58:59.685178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:58:59.685178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:58:59.685178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:58:59.685178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:58:59.685178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:58:59.685178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:58:59.685178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 613/1239] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:58:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:58:59.690031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0293b1c-0863-4bf1-8556-22ced3ead5bd None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:59.801290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0293b1c-0863-4bf1-8556-22ced3ead5bd None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:58:59.837378 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7abe2-1a1b-441a-b793-dc84e7a1d733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:58:59.838401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7abe2-1a1b-441a-b793-dc84e7a1d733 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:58:59.838735 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7abe2-1a1b-441a-b793-dc84e7a1d733 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:58:59.839010 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7abe2-1a1b-441a-b793-dc84e7a1d733 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:58:59.840513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7abe2-1a1b-441a-b793-dc84e7a1d733 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:58:59.840513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7abe2-1a1b-441a-b793-dc84e7a1d733 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:58:59.841701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7abe2-1a1b-441a-b793-dc84e7a1d733 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:58:59.863801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 627/1240] 10.222.0.79 () {56 vars in 1143 bytes} [Mon Jul 4 09:58:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:58:59.869734 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07d592-0786-4d80-b885-bcbe8d83c33b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:58:59.978399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07d592-0786-4d80-b885-bcbe8d83c33b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:00.014783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6653e-ca9b-4791-9479-40096591c5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:00.015427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6653e-ca9b-4791-9479-40096591c5cd None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:00.015711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6653e-ca9b-4791-9479-40096591c5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:00.015985 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6653e-ca9b-4791-9479-40096591c5cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:00.079619 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:00.086843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6653e-ca9b-4791-9479-40096591c5cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:00.088354 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6653e-ca9b-4791-9479-40096591c5cd None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:00.198745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 614/1241] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:58:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:59:00.205378 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7abe2-1a1b-441a-b793-dc84e7a1d733 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:00.236992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:00.314597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7abe2-1a1b-441a-b793-dc84e7a1d733 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:00.351314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25562ddd-a456-4096-9e62-903f4ddc41be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:00.352071 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25562ddd-a456-4096-9e62-903f4ddc41be None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:00.352389 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25562ddd-a456-4096-9e62-903f4ddc41be None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:00.352636 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25562ddd-a456-4096-9e62-903f4ddc41be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:00.365883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25562ddd-a456-4096-9e62-903f4ddc41be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:00.367909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25562ddd-a456-4096-9e62-903f4ddc41be None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-25562ddd-a456-4096-9e62-903f4ddc41be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:59:00.387043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:00.388337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:00.389252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:59:00.389252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:59:00.389252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:00.389252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:00.389252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 628/1242] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:00.395022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6653e-ca9b-4791-9479-40096591c5cd None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:00.497902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:00.504527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6653e-ca9b-4791-9479-40096591c5cd None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:00.539953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08232e-3ac6-4260-ba92-3a3f013f5611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:00.540749 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08232e-3ac6-4260-ba92-3a3f013f5611 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:00.541030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08232e-3ac6-4260-ba92-3a3f013f5611 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:00.541337 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08232e-3ac6-4260-ba92-3a3f013f5611 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:00.542417 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08232e-3ac6-4260-ba92-3a3f013f5611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:00.542758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08232e-3ac6-4260-ba92-3a3f013f5611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:00.544436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08232e-3ac6-4260-ba92-3a3f013f5611 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:00.556601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 615/1243] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:59:00.562825 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25562ddd-a456-4096-9e62-903f4ddc41be None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:00.647709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:00.670200 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25562ddd-a456-4096-9e62-903f4ddc41be None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:00.710355 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509a2b8-e9a2-4327-bddf-396dc9ba62bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:00.711239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509a2b8-e9a2-4327-bddf-396dc9ba62bc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:00.711518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509a2b8-e9a2-4327-bddf-396dc9ba62bc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:00.711792 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509a2b8-e9a2-4327-bddf-396dc9ba62bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:00.719010 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509a2b8-e9a2-4327-bddf-396dc9ba62bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:00.720488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509a2b8-e9a2-4327-bddf-396dc9ba62bc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-7509a2b8-e9a2-4327-bddf-396dc9ba62bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:59:00.736759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:00.737935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:00.739063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:59:00.739063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:59:00.739063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:59:00.739063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:59:00.739063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:59:00.739063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:59:00.739063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:59:00.739063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:00.739063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 629/1244] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:59:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:59:00.744565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08232e-3ac6-4260-ba92-3a3f013f5611 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:00.858375 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08232e-3ac6-4260-ba92-3a3f013f5611 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:00.903612 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1beda3a-cb28-468f-8340-34c3164c3531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:00.904336 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1beda3a-cb28-468f-8340-34c3164c3531 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:00.904503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1beda3a-cb28-468f-8340-34c3164c3531 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:00.904757 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1beda3a-cb28-468f-8340-34c3164c3531 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:00.906238 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1beda3a-cb28-468f-8340-34c3164c3531 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:00.906567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1beda3a-cb28-468f-8340-34c3164c3531 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:00.908063 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1beda3a-cb28-468f-8340-34c3164c3531 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:00.922667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 616/1245] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:59:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:00.928928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509a2b8-e9a2-4327-bddf-396dc9ba62bc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:01.040478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509a2b8-e9a2-4327-bddf-396dc9ba62bc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:01.076128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad46cc-9b9a-4ffd-ad19-c5b2645d0c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:01.076793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad46cc-9b9a-4ffd-ad19-c5b2645d0c49 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:01.077108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad46cc-9b9a-4ffd-ad19-c5b2645d0c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:01.077891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad46cc-9b9a-4ffd-ad19-c5b2645d0c49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:01.078736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad46cc-9b9a-4ffd-ad19-c5b2645d0c49 None admin] RBAC: Adding query filter params (role.id=fc6661dcb3994755a5c238bf37e17bb9, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=5dc48318ab7944c197bf5f7673511931) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:01.079085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad46cc-9b9a-4ffd-ad19-c5b2645d0c49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:01.080504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad46cc-9b9a-4ffd-ad19-c5b2645d0c49 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:01.086609 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 630/1246] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:59:00 2022] GET /identity/v3/role_assignments?user.id=5dc48318ab7944c197bf5f7673511931&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=fc6661dcb3994755a5c238bf37e17bb9 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:03.284107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7a616-4f13-4af5-8f76-6b01fe991d87 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:03.284653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7a616-4f13-4af5-8f76-6b01fe991d87 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:03.284653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7a616-4f13-4af5-8f76-6b01fe991d87 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:03.286132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 617/1247] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:59:03.291863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410888bb-f04d-41e3-8752-83be6d8eb97a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:03.292274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410888bb-f04d-41e3-8752-83be6d8eb97a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:03.292483 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410888bb-f04d-41e3-8752-83be6d8eb97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:03.298306 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:03.313115 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-410888bb-f04d-41e3-8752-83be6d8eb97a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:59:03.314488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-410888bb-f04d-41e3-8752-83be6d8eb97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:59:03.387650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 631/1248] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:59:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:59:03.414342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7a616-4f13-4af5-8f76-6b01fe991d87 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:03.434555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:03.525426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7a616-4f13-4af5-8f76-6b01fe991d87 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:03.562443 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4a23a-71bc-4c3c-a1e8-c55010472fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:03.563225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4a23a-71bc-4c3c-a1e8-c55010472fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:03.563507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4a23a-71bc-4c3c-a1e8-c55010472fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:03.563785 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4a23a-71bc-4c3c-a1e8-c55010472fa1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:03.574235 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4a23a-71bc-4c3c-a1e8-c55010472fa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-5cc4a23a-71bc-4c3c-a1e8-c55010472fa1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:59:03.582073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:03.584039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:03.586030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:03.586030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:03.586030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:03.586030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:03.586030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:59:03.586030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:59:03.586030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:59:03.586030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:59:03.586030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:59:03.586030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:59:03.586030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:59:03.586030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:59:03.586030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 618/1249] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:03.591442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410888bb-f04d-41e3-8752-83be6d8eb97a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:03.705252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410888bb-f04d-41e3-8752-83be6d8eb97a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:03.738064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:03.743716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b27b32-3f4e-46ed-9614-9d02c3f478b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:03.744395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b27b32-3f4e-46ed-9614-9d02c3f478b2 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:03.744679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b27b32-3f4e-46ed-9614-9d02c3f478b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:03.744966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b27b32-3f4e-46ed-9614-9d02c3f478b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:03.745744 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b27b32-3f4e-46ed-9614-9d02c3f478b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:03.746038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b27b32-3f4e-46ed-9614-9d02c3f478b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:03.747435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b27b32-3f4e-46ed-9614-9d02c3f478b2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:03.758325 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 632/1250] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:59:03.764706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4a23a-71bc-4c3c-a1e8-c55010472fa1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:03.855810 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:03.885083 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4a23a-71bc-4c3c-a1e8-c55010472fa1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:03.924427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd061514-e588-46c8-8c90-ebc7f1e00af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:03.925286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd061514-e588-46c8-8c90-ebc7f1e00af4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:03.925658 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd061514-e588-46c8-8c90-ebc7f1e00af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:03.925928 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd061514-e588-46c8-8c90-ebc7f1e00af4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:03.938390 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd061514-e588-46c8-8c90-ebc7f1e00af4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:03.939877 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd061514-e588-46c8-8c90-ebc7f1e00af4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-cd061514-e588-46c8-8c90-ebc7f1e00af4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:59:03.958688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:03.959856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:03.960781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:59:03.960781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:59:03.960781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:03.960781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:59:03.960781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 619/1251] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:03.966194 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b27b32-3f4e-46ed-9614-9d02c3f478b2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:04.076702 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b27b32-3f4e-46ed-9614-9d02c3f478b2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:04.114064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ef6eb-33bb-4228-a71e-0053b014620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:04.114872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ef6eb-33bb-4228-a71e-0053b014620a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:04.115161 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ef6eb-33bb-4228-a71e-0053b014620a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:04.115494 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ef6eb-33bb-4228-a71e-0053b014620a None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:04.116320 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ef6eb-33bb-4228-a71e-0053b014620a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:04.116605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ef6eb-33bb-4228-a71e-0053b014620a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:04.118256 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ef6eb-33bb-4228-a71e-0053b014620a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:04.136965 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 633/1252] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:59:04.142791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd061514-e588-46c8-8c90-ebc7f1e00af4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:04.252496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd061514-e588-46c8-8c90-ebc7f1e00af4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:04.279937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:04.288437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885adf6f-bfb9-4966-a30a-00869d927f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:04.289247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885adf6f-bfb9-4966-a30a-00869d927f40 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:04.289531 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885adf6f-bfb9-4966-a30a-00869d927f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:04.289847 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885adf6f-bfb9-4966-a30a-00869d927f40 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:04.298043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885adf6f-bfb9-4966-a30a-00869d927f40 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:04.299792 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885adf6f-bfb9-4966-a30a-00869d927f40 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-885adf6f-bfb9-4966-a30a-00869d927f40 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:59:04.315257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:04.316492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:04.322420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:04.322420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:04.322420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:59:04.322420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:59:04.322420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:59:04.322420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:59:04.322420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:59:04.322420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:59:04.322420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:59:04.322420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:59:04.322420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 620/1253] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:59:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:04.323304 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ef6eb-33bb-4228-a71e-0053b014620a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:04.385452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:04.437886 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ef6eb-33bb-4228-a71e-0053b014620a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:04.478562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0e6b1-7a53-4446-b77d-35793e6d7f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:04.479473 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0e6b1-7a53-4446-b77d-35793e6d7f75 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:04.479802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0e6b1-7a53-4446-b77d-35793e6d7f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:04.480198 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0e6b1-7a53-4446-b77d-35793e6d7f75 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:04.481346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0e6b1-7a53-4446-b77d-35793e6d7f75 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:04.481692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0e6b1-7a53-4446-b77d-35793e6d7f75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:04.483623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0e6b1-7a53-4446-b77d-35793e6d7f75 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:04.505675 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 634/1254] 10.222.0.79 () {56 vars in 1143 bytes} [Mon Jul 4 09:59:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:04.511468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885adf6f-bfb9-4966-a30a-00869d927f40 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:04.624495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885adf6f-bfb9-4966-a30a-00869d927f40 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:04.663875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fbbc5-174c-4535-aad5-609909964e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:04.664546 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fbbc5-174c-4535-aad5-609909964e6a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:04.664895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fbbc5-174c-4535-aad5-609909964e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:04.665352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fbbc5-174c-4535-aad5-609909964e6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:04.732400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:04.747562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93fbbc5-174c-4535-aad5-609909964e6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:04.749251 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93fbbc5-174c-4535-aad5-609909964e6a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:04.877842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 621/1255] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:59:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:59:04.883947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0e6b1-7a53-4446-b77d-35793e6d7f75 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:04.999110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0e6b1-7a53-4446-b77d-35793e6d7f75 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:05.016867 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:05.039024 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc8a57-43f1-4d2f-b709-e6977c121538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:05.039825 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc8a57-43f1-4d2f-b709-e6977c121538 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:05.040109 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc8a57-43f1-4d2f-b709-e6977c121538 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:05.040384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc8a57-43f1-4d2f-b709-e6977c121538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:05.055812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc8a57-43f1-4d2f-b709-e6977c121538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:05.057312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc8a57-43f1-4d2f-b709-e6977c121538 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-84fc8a57-43f1-4d2f-b709-e6977c121538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:59:05.076523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:05.077695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:05.078762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:59:05.078762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:59:05.078762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:05.078762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:05.078762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 635/1256] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:05.084537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fbbc5-174c-4535-aad5-609909964e6a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:05.182816 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:05.196575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fbbc5-174c-4535-aad5-609909964e6a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:05.234685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ea2b7-08c6-43ee-bac5-411f03218fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:05.235356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ea2b7-08c6-43ee-bac5-411f03218fde None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:05.235645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ea2b7-08c6-43ee-bac5-411f03218fde None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:05.235927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ea2b7-08c6-43ee-bac5-411f03218fde None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:05.236644 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ea2b7-08c6-43ee-bac5-411f03218fde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:05.236953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ea2b7-08c6-43ee-bac5-411f03218fde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:05.238360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ea2b7-08c6-43ee-bac5-411f03218fde None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:05.255577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 622/1257] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:05.257037 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc8a57-43f1-4d2f-b709-e6977c121538 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:05.364533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc8a57-43f1-4d2f-b709-e6977c121538 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:05.404366 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d66fd0-141d-4488-b57c-5844b8997680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:05.405229 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d66fd0-141d-4488-b57c-5844b8997680 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:05.405547 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d66fd0-141d-4488-b57c-5844b8997680 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:05.405972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d66fd0-141d-4488-b57c-5844b8997680 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:05.422152 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d66fd0-141d-4488-b57c-5844b8997680 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:05.424022 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d66fd0-141d-4488-b57c-5844b8997680 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-54d66fd0-141d-4488-b57c-5844b8997680 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:59:05.440549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:05.441746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:05.447923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:59:05.447923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:59:05.447923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:59:05.447923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:59:05.447923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:59:05.447923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:59:05.447923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:59:05.447923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:05.447923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 636/1258] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:59:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:59:05.448543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ea2b7-08c6-43ee-bac5-411f03218fde None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:05.567638 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ea2b7-08c6-43ee-bac5-411f03218fde None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:05.611396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1776e78-03c8-4815-b3e6-e548ef011168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:05.612078 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1776e78-03c8-4815-b3e6-e548ef011168 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:05.612362 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1776e78-03c8-4815-b3e6-e548ef011168 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:05.612642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1776e78-03c8-4815-b3e6-e548ef011168 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:05.613439 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1776e78-03c8-4815-b3e6-e548ef011168 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:05.613811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1776e78-03c8-4815-b3e6-e548ef011168 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:05.615258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1776e78-03c8-4815-b3e6-e548ef011168 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:05.628680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 623/1259] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:59:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:05.635263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d66fd0-141d-4488-b57c-5844b8997680 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:05.663888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:05.742971 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d66fd0-141d-4488-b57c-5844b8997680 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:05.777548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a5b0f-f5ca-4e98-baaa-fe793eb58a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:05.778235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a5b0f-f5ca-4e98-baaa-fe793eb58a5b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:05.778552 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a5b0f-f5ca-4e98-baaa-fe793eb58a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:05.778828 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a5b0f-f5ca-4e98-baaa-fe793eb58a5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:05.779681 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a5b0f-f5ca-4e98-baaa-fe793eb58a5b None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=5dc48318ab7944c197bf5f7673511931) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:05.779965 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a5b0f-f5ca-4e98-baaa-fe793eb58a5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:05.782012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a5b0f-f5ca-4e98-baaa-fe793eb58a5b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:05.787465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 637/1260] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:59:05 2022] GET /identity/v3/role_assignments?user.id=5dc48318ab7944c197bf5f7673511931&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=a3de9df253294d069dc1867fa26600c3 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:07.823412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bcd04-92ad-4993-aea8-6fad12e3904c None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:07.824283 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bcd04-92ad-4993-aea8-6fad12e3904c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:07.824283 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bcd04-92ad-4993-aea8-6fad12e3904c None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:07.826606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 624/1261] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:59:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:59:07.831993 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286a012-2267-4bac-9648-98424271691a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:07.832438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286a012-2267-4bac-9648-98424271691a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:07.832529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286a012-2267-4bac-9648-98424271691a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:07.852621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0286a012-2267-4bac-9648-98424271691a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:59:07.854002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0286a012-2267-4bac-9648-98424271691a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:59:07.928877 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 638/1262] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:59:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:59:07.951778 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bcd04-92ad-4993-aea8-6fad12e3904c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:07.963588 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:08.063102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bcd04-92ad-4993-aea8-6fad12e3904c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:08.101135 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede95d3-e54d-488a-b037-211d17f9373d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:08.101879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede95d3-e54d-488a-b037-211d17f9373d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:08.102163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede95d3-e54d-488a-b037-211d17f9373d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:08.102439 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede95d3-e54d-488a-b037-211d17f9373d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:08.114173 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede95d3-e54d-488a-b037-211d17f9373d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8ede95d3-e54d-488a-b037-211d17f9373d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:59:08.121783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:08.122906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:08.123754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:08.123754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:08.123754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:08.123754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:08.123754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:59:08.123754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:59:08.123754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:59:08.123754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:59:08.123754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:59:08.123754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:59:08.123754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:59:08.123754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:59:08.123754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 625/1263] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:59:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:08.130813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286a012-2267-4bac-9648-98424271691a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:08.152555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:08.239933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286a012-2267-4bac-9648-98424271691a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:08.278097 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79b455-b43f-4fde-a8df-a978799942b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:08.278754 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79b455-b43f-4fde-a8df-a978799942b8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:08.279036 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79b455-b43f-4fde-a8df-a978799942b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:08.279347 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79b455-b43f-4fde-a8df-a978799942b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:08.280400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79b455-b43f-4fde-a8df-a978799942b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:08.280547 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79b455-b43f-4fde-a8df-a978799942b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:08.281964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79b455-b43f-4fde-a8df-a978799942b8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:08.290254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 639/1264] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:59:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:08.300799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede95d3-e54d-488a-b037-211d17f9373d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:08.374937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:08.413630 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede95d3-e54d-488a-b037-211d17f9373d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:08.451197 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4db9e0-51d8-497d-82ff-55dc15695381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:08.452161 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4db9e0-51d8-497d-82ff-55dc15695381 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:08.452752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4db9e0-51d8-497d-82ff-55dc15695381 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:08.453028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4db9e0-51d8-497d-82ff-55dc15695381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:08.464171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4db9e0-51d8-497d-82ff-55dc15695381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:08.465649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4db9e0-51d8-497d-82ff-55dc15695381 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-bd4db9e0-51d8-497d-82ff-55dc15695381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:59:08.484266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:08.485295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:08.486257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:59:08.486257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:59:08.486257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:08.486257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:59:08.486257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 626/1265] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:08.494105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79b455-b43f-4fde-a8df-a978799942b8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:08.582647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:08.604261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79b455-b43f-4fde-a8df-a978799942b8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:08.638941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2395e6-ca8e-48d5-97e8-9751c7f71351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:08.639526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2395e6-ca8e-48d5-97e8-9751c7f71351 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:08.639816 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2395e6-ca8e-48d5-97e8-9751c7f71351 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:08.640136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2395e6-ca8e-48d5-97e8-9751c7f71351 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:08.641164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2395e6-ca8e-48d5-97e8-9751c7f71351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:08.641480 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2395e6-ca8e-48d5-97e8-9751c7f71351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:08.642884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2395e6-ca8e-48d5-97e8-9751c7f71351 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:08.655282 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 640/1266] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:08.664383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4db9e0-51d8-497d-82ff-55dc15695381 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:08.776457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4db9e0-51d8-497d-82ff-55dc15695381 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:08.798835 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:08.822062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4149b90-cfa6-417f-94fc-a35e25e32ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:08.823270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4149b90-cfa6-417f-94fc-a35e25e32ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:08.823679 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4149b90-cfa6-417f-94fc-a35e25e32ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:08.824266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4149b90-cfa6-417f-94fc-a35e25e32ee4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:08.835975 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4149b90-cfa6-417f-94fc-a35e25e32ee4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:08.837643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4149b90-cfa6-417f-94fc-a35e25e32ee4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b4149b90-cfa6-417f-94fc-a35e25e32ee4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:59:08.856361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:08.857404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:08.858680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:08.858680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:08.858680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:59:08.858680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:59:08.858680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:59:08.858680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:59:08.858680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:59:08.858680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:59:08.858680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:59:08.858680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:59:08.858680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 627/1267] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:59:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:08.865290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2395e6-ca8e-48d5-97e8-9751c7f71351 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:08.979084 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2395e6-ca8e-48d5-97e8-9751c7f71351 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:09.017532 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb54e0-784b-4ea6-9490-a90a209e3db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:09.018227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb54e0-784b-4ea6-9490-a90a209e3db4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:09.018507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb54e0-784b-4ea6-9490-a90a209e3db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:09.018787 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb54e0-784b-4ea6-9490-a90a209e3db4 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:09.019626 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb54e0-784b-4ea6-9490-a90a209e3db4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:09.019933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb54e0-784b-4ea6-9490-a90a209e3db4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:09.021390 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb54e0-784b-4ea6-9490-a90a209e3db4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:09.030985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:09.040420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 641/1268] 10.222.0.79 () {56 vars in 1143 bytes} [Mon Jul 4 09:59:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:09.046752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4149b90-cfa6-417f-94fc-a35e25e32ee4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:09.154779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4149b90-cfa6-417f-94fc-a35e25e32ee4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:09.191638 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354dbaf5-777c-4672-863b-53795b3bd7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:09.192425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354dbaf5-777c-4672-863b-53795b3bd7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:09.192706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354dbaf5-777c-4672-863b-53795b3bd7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:09.192982 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354dbaf5-777c-4672-863b-53795b3bd7f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:09.203921 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354dbaf5-777c-4672-863b-53795b3bd7f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:09.205396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354dbaf5-777c-4672-863b-53795b3bd7f0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-354dbaf5-777c-4672-863b-53795b3bd7f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:59:09.222963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:09.224015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:09.225085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:59:09.225085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:59:09.225085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:09.225085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:59:09.225085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 628/1269] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:59:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:09.230285 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb54e0-784b-4ea6-9490-a90a209e3db4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:09.337210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb54e0-784b-4ea6-9490-a90a209e3db4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:09.373743 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413ea7f-fae7-49a0-964b-7465acb0fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:09.374404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413ea7f-fae7-49a0-964b-7465acb0fa82 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:09.374687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413ea7f-fae7-49a0-964b-7465acb0fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:09.375037 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413ea7f-fae7-49a0-964b-7465acb0fa82 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:09.375779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413ea7f-fae7-49a0-964b-7465acb0fa82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:09.376089 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413ea7f-fae7-49a0-964b-7465acb0fa82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:09.377472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413ea7f-fae7-49a0-964b-7465acb0fa82 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:09.388742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 642/1270] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:09.394860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354dbaf5-777c-4672-863b-53795b3bd7f0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:09.431869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:09.502291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354dbaf5-777c-4672-863b-53795b3bd7f0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:09.537344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e444ec3-4daa-4272-bae4-b846586ee448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:09.538470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e444ec3-4daa-4272-bae4-b846586ee448 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:09.538756 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e444ec3-4daa-4272-bae4-b846586ee448 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:09.539034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e444ec3-4daa-4272-bae4-b846586ee448 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:09.545922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e444ec3-4daa-4272-bae4-b846586ee448 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:09.548142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e444ec3-4daa-4272-bae4-b846586ee448 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-4e444ec3-4daa-4272-bae4-b846586ee448 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:59:09.562849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:09.563853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:09.564811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:59:09.564811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:59:09.564811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:59:09.564811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:59:09.564811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:59:09.564811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:59:09.564811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:59:09.564811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:59:09.564811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 629/1271] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:59:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:09.570012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413ea7f-fae7-49a0-964b-7465acb0fa82 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:09.629529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:09.671811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413ea7f-fae7-49a0-964b-7465acb0fa82 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:09.705817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca046c8-46b6-41f0-a740-cc7ba70c05f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:09.706487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca046c8-46b6-41f0-a740-cc7ba70c05f8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:09.706775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca046c8-46b6-41f0-a740-cc7ba70c05f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:09.707048 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca046c8-46b6-41f0-a740-cc7ba70c05f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:09.707838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca046c8-46b6-41f0-a740-cc7ba70c05f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:09.708148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca046c8-46b6-41f0-a740-cc7ba70c05f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:09.709592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca046c8-46b6-41f0-a740-cc7ba70c05f8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:09.721956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 643/1272] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:09.728569 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e444ec3-4daa-4272-bae4-b846586ee448 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:09.819129 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:09.834284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e444ec3-4daa-4272-bae4-b846586ee448 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:09.876817 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb15454-6b88-4d01-95aa-7697dac66f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:09.877831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb15454-6b88-4d01-95aa-7697dac66f0e None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:09.878187 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb15454-6b88-4d01-95aa-7697dac66f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:09.878465 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb15454-6b88-4d01-95aa-7697dac66f0e None admin] PATH_INFO: `/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/5dc48318ab7944c197bf5f7673511931/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:09.880952 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb15454-6b88-4d01-95aa-7697dac66f0e None admin] RBAC: Authorizing `identity:create_grant(project_id=f32306262e8744eab4bdc6d28e24e6d0, user_id=5dc48318ab7944c197bf5f7673511931, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:09.882575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb15454-6b88-4d01-95aa-7697dac66f0e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:09.897953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 630/1273] 10.222.0.79 () {58 vars in 1439 bytes} [Mon Jul 4 09:59:09 2022] PUT /identity/v3/projects/f32306262e8744eab4bdc6d28e24e6d0/users/5dc48318ab7944c197bf5f7673511931/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:59:12.026068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e735e-04ca-4b20-ba2c-64c9d867f407 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:12.027029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e735e-04ca-4b20-ba2c-64c9d867f407 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:12.027029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e735e-04ca-4b20-ba2c-64c9d867f407 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:12.029127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 644/1274] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:59:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:59:12.034551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78583ff1-9fd5-43b0-af13-287fd844e91a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:12.034984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78583ff1-9fd5-43b0-af13-287fd844e91a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:12.035076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78583ff1-9fd5-43b0-af13-287fd844e91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:12.053127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-78583ff1-9fd5-43b0-af13-287fd844e91a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:59:12.054353 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-78583ff1-9fd5-43b0-af13-287fd844e91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:59:12.124351 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 631/1275] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:59:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:59:12.152349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e735e-04ca-4b20-ba2c-64c9d867f407 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:12.271229 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e735e-04ca-4b20-ba2c-64c9d867f407 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:12.289332 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:12.311641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c93a08-1091-4082-a771-69c77b0c684e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:12.312733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c93a08-1091-4082-a771-69c77b0c684e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:12.313021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c93a08-1091-4082-a771-69c77b0c684e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:12.313303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c93a08-1091-4082-a771-69c77b0c684e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:12.323633 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c93a08-1091-4082-a771-69c77b0c684e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-83c93a08-1091-4082-a771-69c77b0c684e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:59:12.331596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:12.332655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:12.333815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:12.333815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:12.333815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:12.333815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:12.333815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:59:12.333815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:59:12.333815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:59:12.333815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:59:12.333815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:59:12.333815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:59:12.333815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:59:12.333815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:12.337809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 645/1276] 10.222.0.79 () {56 vars in 1096 bytes} [Mon Jul 4 09:59:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:12.339451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78583ff1-9fd5-43b0-af13-287fd844e91a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:12.435880 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:12.464905 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78583ff1-9fd5-43b0-af13-287fd844e91a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:12.502368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e59563-1966-485e-856f-ce2dd44ecc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:12.503030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e59563-1966-485e-856f-ce2dd44ecc9c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:12.503306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e59563-1966-485e-856f-ce2dd44ecc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:12.503589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e59563-1966-485e-856f-ce2dd44ecc9c None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:12.508529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e59563-1966-485e-856f-ce2dd44ecc9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:12.508529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e59563-1966-485e-856f-ce2dd44ecc9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:12.508529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e59563-1966-485e-856f-ce2dd44ecc9c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:12.515024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 632/1277] 10.222.0.79 () {56 vars in 1099 bytes} [Mon Jul 4 09:59:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:59:12.523394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c93a08-1091-4082-a771-69c77b0c684e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:12.627976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c93a08-1091-4082-a771-69c77b0c684e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:12.669148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27316d41-a161-4354-b522-4c6820d596f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:12.670531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27316d41-a161-4354-b522-4c6820d596f8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:12.670960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27316d41-a161-4354-b522-4c6820d596f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:12.671354 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27316d41-a161-4354-b522-4c6820d596f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:12.682748 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27316d41-a161-4354-b522-4c6820d596f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:12.684202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27316d41-a161-4354-b522-4c6820d596f8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-27316d41-a161-4354-b522-4c6820d596f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:59:12.701417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:12.702560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:12.703727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:59:12.703727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:59:12.703727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:12.703727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:12.703727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 646/1278] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:59:12.708714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e59563-1966-485e-856f-ce2dd44ecc9c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:12.825464 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e59563-1966-485e-856f-ce2dd44ecc9c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:12.869388 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05c3fc-3eba-4005-ab13-bdcdda6f8a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:12.870125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05c3fc-3eba-4005-ab13-bdcdda6f8a3d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:12.870585 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05c3fc-3eba-4005-ab13-bdcdda6f8a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:12.870998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05c3fc-3eba-4005-ab13-bdcdda6f8a3d None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:12.871782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b05c3fc-3eba-4005-ab13-bdcdda6f8a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:12.872294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b05c3fc-3eba-4005-ab13-bdcdda6f8a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:12.874175 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b05c3fc-3eba-4005-ab13-bdcdda6f8a3d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:12.889196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 633/1279] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:12.895297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27316d41-a161-4354-b522-4c6820d596f8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:12.926588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:12.998024 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27316d41-a161-4354-b522-4c6820d596f8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:13.032980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfe1c1-3ee0-4851-99fc-df972d5803b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:13.033782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfe1c1-3ee0-4851-99fc-df972d5803b5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:13.034090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfe1c1-3ee0-4851-99fc-df972d5803b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:13.034367 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfe1c1-3ee0-4851-99fc-df972d5803b5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:13.041936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfe1c1-3ee0-4851-99fc-df972d5803b5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:13.043394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfe1c1-3ee0-4851-99fc-df972d5803b5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-1ddfe1c1-3ee0-4851-99fc-df972d5803b5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:59:13.057664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:13.058605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:13.059570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:13.059570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:13.059570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:59:13.059570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:59:13.059570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:59:13.059570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:59:13.059570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:59:13.059570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:59:13.059570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:59:13.059570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:13.059570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 647/1280] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:59:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:13.064943 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05c3fc-3eba-4005-ab13-bdcdda6f8a3d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:13.070815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:13.176274 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05c3fc-3eba-4005-ab13-bdcdda6f8a3d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:13.212093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd8da9-348c-4931-9931-32d9feab45b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:13.212755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd8da9-348c-4931-9931-32d9feab45b9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:13.213036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd8da9-348c-4931-9931-32d9feab45b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:13.213307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd8da9-348c-4931-9931-32d9feab45b9 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:13.214176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd8da9-348c-4931-9931-32d9feab45b9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:13.214484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd8da9-348c-4931-9931-32d9feab45b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:13.216324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd8da9-348c-4931-9931-32d9feab45b9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:13.234106 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 634/1281] 10.222.0.79 () {56 vars in 1143 bytes} [Mon Jul 4 09:59:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:13.240085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfe1c1-3ee0-4851-99fc-df972d5803b5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:13.340202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:13.351342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfe1c1-3ee0-4851-99fc-df972d5803b5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:13.394099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8dd8ff-4975-4845-a2d1-c10c7580dad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:13.394748 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dd8ff-4975-4845-a2d1-c10c7580dad4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:13.395015 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dd8ff-4975-4845-a2d1-c10c7580dad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:13.395311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dd8ff-4975-4845-a2d1-c10c7580dad4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:13.465228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8dd8ff-4975-4845-a2d1-c10c7580dad4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:13.466652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8dd8ff-4975-4845-a2d1-c10c7580dad4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:13.561808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:13.574001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 648/1282] 10.222.0.79 () {58 vars in 1282 bytes} [Mon Jul 4 09:59:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:59:13.580620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd8da9-348c-4931-9931-32d9feab45b9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:13.649745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:13.687997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd8da9-348c-4931-9931-32d9feab45b9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:13.723717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fbaa1-fd18-451c-92a9-f1ae045bf764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:13.724433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fbaa1-fd18-451c-92a9-f1ae045bf764 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:13.724709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fbaa1-fd18-451c-92a9-f1ae045bf764 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:13.724975 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fbaa1-fd18-451c-92a9-f1ae045bf764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:13.738714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812fbaa1-fd18-451c-92a9-f1ae045bf764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:13.740179 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812fbaa1-fd18-451c-92a9-f1ae045bf764 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-812fbaa1-fd18-451c-92a9-f1ae045bf764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:59:13.758275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:13.759270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:13.760311 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:59:13.760311 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:59:13.760311 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:59:13.760311 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:59:13.760311 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 635/1283] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 09:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:13.766193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8dd8ff-4975-4845-a2d1-c10c7580dad4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:13.891894 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8dd8ff-4975-4845-a2d1-c10c7580dad4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:13.930875 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11252a52-4d11-4147-b2ab-8e63810b5dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:13.931594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11252a52-4d11-4147-b2ab-8e63810b5dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:13.931852 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11252a52-4d11-4147-b2ab-8e63810b5dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:13.932125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11252a52-4d11-4147-b2ab-8e63810b5dd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:13.932857 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11252a52-4d11-4147-b2ab-8e63810b5dd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:13.933165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11252a52-4d11-4147-b2ab-8e63810b5dd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:13.934581 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11252a52-4d11-4147-b2ab-8e63810b5dd5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:13.946486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 649/1284] 10.222.0.79 () {56 vars in 1109 bytes} [Mon Jul 4 09:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:13.955058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fbaa1-fd18-451c-92a9-f1ae045bf764 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:14.064065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fbaa1-fd18-451c-92a9-f1ae045bf764 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:14.078516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:14.102024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226df22-fc76-4044-b863-347e5c8b6c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:14.102805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226df22-fc76-4044-b863-347e5c8b6c60 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:14.103084 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226df22-fc76-4044-b863-347e5c8b6c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:14.103342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226df22-fc76-4044-b863-347e5c8b6c60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:14.110608 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226df22-fc76-4044-b863-347e5c8b6c60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:14.112067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226df22-fc76-4044-b863-347e5c8b6c60 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-3226df22-fc76-4044-b863-347e5c8b6c60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:59:14.126400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:14.127304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:14.128313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:59:14.128313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:59:14.128313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:59:14.128313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:59:14.128313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:59:14.128313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:59:14.128313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:59:14.128313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jul 04 09:59:14.128313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 636/1285] 10.222.0.79 () {56 vars in 1111 bytes} [Mon Jul 4 09:59:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:14.133767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11252a52-4d11-4147-b2ab-8e63810b5dd5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:14.189164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:14.245649 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11252a52-4d11-4147-b2ab-8e63810b5dd5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:14.281288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f66bcc-e58a-4b15-8d29-e041ae7f2546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:14.281977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f66bcc-e58a-4b15-8d29-e041ae7f2546 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:14.282266 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f66bcc-e58a-4b15-8d29-e041ae7f2546 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:14.282536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f66bcc-e58a-4b15-8d29-e041ae7f2546 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:14.283319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f66bcc-e58a-4b15-8d29-e041ae7f2546 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:14.283624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f66bcc-e58a-4b15-8d29-e041ae7f2546 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:14.285070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f66bcc-e58a-4b15-8d29-e041ae7f2546 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:14.297589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 650/1286] 10.222.0.79 () {56 vars in 1148 bytes} [Mon Jul 4 09:59:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:14.303939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226df22-fc76-4044-b863-347e5c8b6c60 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:14.409567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226df22-fc76-4044-b863-347e5c8b6c60 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:14.443401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57659c-cd39-4712-877b-e41a9669e22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:14.444043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57659c-cd39-4712-877b-e41a9669e22d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:14.444322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57659c-cd39-4712-877b-e41a9669e22d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:14.444595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57659c-cd39-4712-877b-e41a9669e22d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:14.445413 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57659c-cd39-4712-877b-e41a9669e22d None admin] RBAC: Adding query filter params (role.id=a3de9df253294d069dc1867fa26600c3, scope.project.id=f32306262e8744eab4bdc6d28e24e6d0, user.id=5dc48318ab7944c197bf5f7673511931) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:14.445769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57659c-cd39-4712-877b-e41a9669e22d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:14.447215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57659c-cd39-4712-877b-e41a9669e22d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:14.453325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 637/1287] 10.222.0.79 () {56 vars in 1374 bytes} [Mon Jul 4 09:59:14 2022] GET /identity/v3/role_assignments?user.id=5dc48318ab7944c197bf5f7673511931&scope.project.id=f32306262e8744eab4bdc6d28e24e6d0&role.id=a3de9df253294d069dc1867fa26600c3 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:16.492617 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb3a09-4f5a-40c0-ba23-81594128afc5 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:16.493179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb3a09-4f5a-40c0-ba23-81594128afc5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:16.493179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb3a09-4f5a-40c0-ba23-81594128afc5 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:16.494462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 651/1288] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:59:16.501546 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95600ee-62ee-4154-a097-0d27ed81c7cb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:16.502352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95600ee-62ee-4154-a097-0d27ed81c7cb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:16.502515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95600ee-62ee-4154-a097-0d27ed81c7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:16.525919 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e95600ee-62ee-4154-a097-0d27ed81c7cb None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:59:16.527115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e95600ee-62ee-4154-a097-0d27ed81c7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:59:16.559630 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:16.603362 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 638/1289] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:59:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:59:16.628711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb3a09-4f5a-40c0-ba23-81594128afc5 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:16.750150 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb3a09-4f5a-40c0-ba23-81594128afc5 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:16.760817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:16.786165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8808a5-34a6-4dc0-83e3-2610c3bc1cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:16.786964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8808a5-34a6-4dc0-83e3-2610c3bc1cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:16.787302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8808a5-34a6-4dc0-83e3-2610c3bc1cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:16.787577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8808a5-34a6-4dc0-83e3-2610c3bc1cc0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:16.788232 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8808a5-34a6-4dc0-83e3-2610c3bc1cc0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:16.789612 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8808a5-34a6-4dc0-83e3-2610c3bc1cc0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-7e8808a5-34a6-4dc0-83e3-2610c3bc1cc0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:16.801818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:16.802982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:16.804286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 652/1290] 10.222.0.79 () {56 vars in 1117 bytes} [Mon Jul 4 09:59:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:16.811563 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95600ee-62ee-4154-a097-0d27ed81c7cb None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:16.935762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95600ee-62ee-4154-a097-0d27ed81c7cb None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:16.972738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9256e8-ea25-4698-b685-10c51751abe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:16.973397 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9256e8-ea25-4698-b685-10c51751abe7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:16.973670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9256e8-ea25-4698-b685-10c51751abe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:16.973985 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9256e8-ea25-4698-b685-10c51751abe7 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:16.974693 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9256e8-ea25-4698-b685-10c51751abe7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:16.975002 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9256e8-ea25-4698-b685-10c51751abe7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:16.976395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9256e8-ea25-4698-b685-10c51751abe7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:16.981376 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 639/1291] 10.222.0.79 () {56 vars in 1116 bytes} [Mon Jul 4 09:59:16 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:16.988182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8808a5-34a6-4dc0-83e3-2610c3bc1cc0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:17.091188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8808a5-34a6-4dc0-83e3-2610c3bc1cc0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:17.128668 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f4a06-2fd5-40e7-925f-580e71244be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:17.129695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f4a06-2fd5-40e7-925f-580e71244be5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:17.130071 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f4a06-2fd5-40e7-925f-580e71244be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:17.130389 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f4a06-2fd5-40e7-925f-580e71244be5 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:17.131400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f4a06-2fd5-40e7-925f-580e71244be5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:17.131868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f4a06-2fd5-40e7-925f-580e71244be5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:17.134075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f4a06-2fd5-40e7-925f-580e71244be5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:17.142200 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 653/1292] 10.222.0.79 () {56 vars in 1116 bytes} [Mon Jul 4 09:59:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:19.225184 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcae8d-67c2-4036-bdfd-1b29fd455ad4 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:19.226180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcae8d-67c2-4036-bdfd-1b29fd455ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:19.226180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcae8d-67c2-4036-bdfd-1b29fd455ad4 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:19.228325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 640/1293] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:59:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:59:19.233877 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877dbf2-2505-406a-ae86-0dccc5435f94 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:19.234410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877dbf2-2505-406a-ae86-0dccc5435f94 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:19.234513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877dbf2-2505-406a-ae86-0dccc5435f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:19.253101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b877dbf2-2505-406a-ae86-0dccc5435f94 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:59:19.254317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b877dbf2-2505-406a-ae86-0dccc5435f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:59:19.286388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:19.324692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 654/1294] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:59:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:59:19.352451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcae8d-67c2-4036-bdfd-1b29fd455ad4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:19.387332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:19.466250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcae8d-67c2-4036-bdfd-1b29fd455ad4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:19.512278 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c75fb-1bd7-4e1c-8b9e-5bc0de7da4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:19.513288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c75fb-1bd7-4e1c-8b9e-5bc0de7da4b1 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:19.513737 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c75fb-1bd7-4e1c-8b9e-5bc0de7da4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:19.514194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c75fb-1bd7-4e1c-8b9e-5bc0de7da4b1 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:19.515281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c75fb-1bd7-4e1c-8b9e-5bc0de7da4b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:19.516668 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c75fb-1bd7-4e1c-8b9e-5bc0de7da4b1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:19.532667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 641/1295] 10.222.0.79 () {60 vars in 1141 bytes} [Mon Jul 4 09:59:19 2022] POST /identity/v3/services => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:59:21.571539 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ae83-63b9-4acb-a659-ac528b85c3e3 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:21.572417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ae83-63b9-4acb-a659-ac528b85c3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:21.572417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ae83-63b9-4acb-a659-ac528b85c3e3 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:21.574624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 655/1296] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:59:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:59:21.579943 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9124bc8-ccb2-4b02-997f-0aaad073cda4 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:21.580364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9124bc8-ccb2-4b02-997f-0aaad073cda4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:21.580483 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9124bc8-ccb2-4b02-997f-0aaad073cda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:21.599116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b9124bc8-ccb2-4b02-997f-0aaad073cda4 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:59:21.600308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b9124bc8-ccb2-4b02-997f-0aaad073cda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:59:21.683885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 642/1297] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:59:21 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:59:21.708821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5ae83-63b9-4acb-a659-ac528b85c3e3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:21.824196 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5ae83-63b9-4acb-a659-ac528b85c3e3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:21.867141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959ba03-46b6-48c2-bf50-30d04d0be1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:21.867948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959ba03-46b6-48c2-bf50-30d04d0be1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:21.868231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959ba03-46b6-48c2-bf50-30d04d0be1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:21.868583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959ba03-46b6-48c2-bf50-30d04d0be1c9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:21.869242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959ba03-46b6-48c2-bf50-30d04d0be1c9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:21.870831 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959ba03-46b6-48c2-bf50-30d04d0be1c9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-4959ba03-46b6-48c2-bf50-30d04d0be1c9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:21.884066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:21.885193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:21.886339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 656/1298] 10.222.0.79 () {56 vars in 1117 bytes} [Mon Jul 4 09:59:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:21.893454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9124bc8-ccb2-4b02-997f-0aaad073cda4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:21.912824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:21.998660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9124bc8-ccb2-4b02-997f-0aaad073cda4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:22.040989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a131b6-0262-4929-8f47-45f40a0fb497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:22.041666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a131b6-0262-4929-8f47-45f40a0fb497 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:22.041945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a131b6-0262-4929-8f47-45f40a0fb497 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:22.042254 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a131b6-0262-4929-8f47-45f40a0fb497 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:22.042958 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a131b6-0262-4929-8f47-45f40a0fb497 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:22.043288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a131b6-0262-4929-8f47-45f40a0fb497 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:22.044713 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a131b6-0262-4929-8f47-45f40a0fb497 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:22.050053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 643/1299] 10.222.0.79 () {56 vars in 1116 bytes} [Mon Jul 4 09:59:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:22.056403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959ba03-46b6-48c2-bf50-30d04d0be1c9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:22.092322 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:22.169449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959ba03-46b6-48c2-bf50-30d04d0be1c9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:22.204255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1fa47-9491-463a-970c-3525d1277fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:22.204903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1fa47-9491-463a-970c-3525d1277fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:22.205187 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1fa47-9491-463a-970c-3525d1277fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:22.205469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1fa47-9491-463a-970c-3525d1277fa6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:22.206286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1fa47-9491-463a-970c-3525d1277fa6 None admin] RBAC: Adding query filter params (interface=public, service_id=f5de74ed8c8348e99ec255da0bb02d1a, region_id=RegionOne) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:22.206601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1fa47-9491-463a-970c-3525d1277fa6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:22.207982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1fa47-9491-463a-970c-3525d1277fa6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:22.213124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 657/1300] 10.222.0.79 () {56 vars in 1251 bytes} [Mon Jul 4 09:59:22 2022] GET /identity/v3/endpoints?service_id=f5de74ed8c8348e99ec255da0bb02d1a&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:22.221100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a131b6-0262-4929-8f47-45f40a0fb497 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:22.303682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:22.323161 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a131b6-0262-4929-8f47-45f40a0fb497 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:22.371199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b3e63-a0a4-4539-9f97-23e62f5c1f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:22.372078 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b3e63-a0a4-4539-9f97-23e62f5c1f75 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:22.372495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b3e63-a0a4-4539-9f97-23e62f5c1f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:22.372906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b3e63-a0a4-4539-9f97-23e62f5c1f75 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:22.373992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b3e63-a0a4-4539-9f97-23e62f5c1f75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:22.376262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b3e63-a0a4-4539-9f97-23e62f5c1f75 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:22.383468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 644/1301] 10.222.0.79 () {56 vars in 1087 bytes} [Mon Jul 4 09:59:22 2022] GET /identity/v3/services => generated 2191 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:59:24.393317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763fce2-e2c2-4b70-9aaa-180fd5bb8ed2 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:24.394202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763fce2-e2c2-4b70-9aaa-180fd5bb8ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:24.394202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763fce2-e2c2-4b70-9aaa-180fd5bb8ed2 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:24.396203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 658/1302] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 09:59:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:59:24.404187 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfc139-e8dd-41b7-b233-d2f9185d1520 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:24.404645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfc139-e8dd-41b7-b233-d2f9185d1520 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:24.404745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfc139-e8dd-41b7-b233-d2f9185d1520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:24.423111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-55bfc139-e8dd-41b7-b233-d2f9185d1520 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:59:24.424288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-55bfc139-e8dd-41b7-b233-d2f9185d1520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:59:24.495300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 645/1303] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 09:59:24 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:59:24.519730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763fce2-e2c2-4b70-9aaa-180fd5bb8ed2 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:24.625603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:24.634390 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763fce2-e2c2-4b70-9aaa-180fd5bb8ed2 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:24.669495 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c0f15-303d-4e6f-b5a4-c5142e5179e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:24.670594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c0f15-303d-4e6f-b5a4-c5142e5179e0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:24.670967 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c0f15-303d-4e6f-b5a4-c5142e5179e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:24.671342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c0f15-303d-4e6f-b5a4-c5142e5179e0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:24.672101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c0f15-303d-4e6f-b5a4-c5142e5179e0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:24.673477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c0f15-303d-4e6f-b5a4-c5142e5179e0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-df2c0f15-303d-4e6f-b5a4-c5142e5179e0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:59:24.685540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:59:24.686691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jul 04 09:59:24.688136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 659/1304] 10.222.0.79 () {56 vars in 1117 bytes} [Mon Jul 4 09:59:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:59:24.692916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfc139-e8dd-41b7-b233-d2f9185d1520 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:24.778938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:24.809812 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfc139-e8dd-41b7-b233-d2f9185d1520 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:24.855541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48cc43-4193-49a6-b6fc-9f593dce166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:24.856204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48cc43-4193-49a6-b6fc-9f593dce166b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:24.856485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48cc43-4193-49a6-b6fc-9f593dce166b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:24.856747 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48cc43-4193-49a6-b6fc-9f593dce166b None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:24.857449 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48cc43-4193-49a6-b6fc-9f593dce166b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:59:24.857790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48cc43-4193-49a6-b6fc-9f593dce166b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:24.859743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48cc43-4193-49a6-b6fc-9f593dce166b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:24.865577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 646/1305] 10.222.0.79 () {56 vars in 1116 bytes} [Mon Jul 4 09:59:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:59:24.871481 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c0f15-303d-4e6f-b5a4-c5142e5179e0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:59:24.989778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c0f15-303d-4e6f-b5a4-c5142e5179e0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:59:25.024966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b433de-7178-4d38-8015-54322a668051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:59:25.025610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b433de-7178-4d38-8015-54322a668051 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:59:25.025947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b433de-7178-4d38-8015-54322a668051 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:59:25.026220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b433de-7178-4d38-8015-54322a668051 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:59:25.026949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b433de-7178-4d38-8015-54322a668051 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:59:25.028326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b433de-7178-4d38-8015-54322a668051 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:59:25.049463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 660/1306] 10.222.0.79 () {60 vars in 1144 bytes} [Mon Jul 4 09:59:24 2022] POST /identity/v3/endpoints => generated 331 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:59:34.861942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=30 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:59:35.045148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:28.382320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbfb0c-2bf9-460f-b269-0063c449d280 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:28.382881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbfb0c-2bf9-460f-b269-0063c449d280 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:28.382881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbfb0c-2bf9-460f-b269-0063c449d280 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:28.385312 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 647/1307] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:00:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:00:28.391155 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c964ec-2b6e-4ebf-9fcd-6ea631ea5507 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:28.391663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c964ec-2b6e-4ebf-9fcd-6ea631ea5507 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:28.391747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c964ec-2b6e-4ebf-9fcd-6ea631ea5507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:28.560856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f3c964ec-2b6e-4ebf-9fcd-6ea631ea5507 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:00:28.562103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f3c964ec-2b6e-4ebf-9fcd-6ea631ea5507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:28.798027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 661/1308] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:00:28.851983 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f361-7bed-4036-844a-18f2633b4b56 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:28.852280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f361-7bed-4036-844a-18f2633b4b56 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:28.852705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f361-7bed-4036-844a-18f2633b4b56 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:28.854364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 648/1309] 10.222.0.79 () {54 vars in 964 bytes} [Mon Jul 4 10:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:00:28.867933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccc681-7279-483f-8cda-e10fa7b677d5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:28.868229 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccc681-7279-483f-8cda-e10fa7b677d5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:28.868518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccc681-7279-483f-8cda-e10fa7b677d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:29.041011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-6eccc681-7279-483f-8cda-e10fa7b677d5 None None] MFA Rules not processed for user `3a1f11a4b8554a98b7626a82f01888c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:00:29.042364 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6eccc681-7279-483f-8cda-e10fa7b677d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:29.090052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:29.239118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 662/1310] 10.222.0.79 () {58 vars in 1061 bytes} [Mon Jul 4 10:00:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:00:29.239496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22f361-7bed-4036-844a-18f2633b4b56 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:00:29.241182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-dd22f361-7bed-4036-844a-18f2633b4b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:29.401599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22f361-7bed-4036-844a-18f2633b4b56 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:00:29.491103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b688464-2bba-4d8a-bb72-97ce96f4f72d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:00:29.492183 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b688464-2bba-4d8a-bb72-97ce96f4f72d service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:29.492698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b688464-2bba-4d8a-bb72-97ce96f4f72d service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:29.493290 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b688464-2bba-4d8a-bb72-97ce96f4f72d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:29.495343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5b688464-2bba-4d8a-bb72-97ce96f4f72d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:29.555244 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:29.621585 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b688464-2bba-4d8a-bb72-97ce96f4f72d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:00:29.623820 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b688464-2bba-4d8a-bb72-97ce96f4f72d service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:00:29.649025 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 649/1311] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:00:32.016022 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160749d-7fe2-43f9-a90e-2a702762b46a None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:32.016673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160749d-7fe2-43f9-a90e-2a702762b46a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:32.016673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160749d-7fe2-43f9-a90e-2a702762b46a None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:32.018159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 663/1312] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:00:32.024307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3ff47-c951-47b1-9107-daa6953cdca7 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:32.024771 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3ff47-c951-47b1-9107-daa6953cdca7 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:32.024864 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3ff47-c951-47b1-9107-daa6953cdca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:32.104789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c6c3ff47-c951-47b1-9107-daa6953cdca7 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:00:32.106058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c6c3ff47-c951-47b1-9107-daa6953cdca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:32.207320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 650/1313] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:00:32.251473 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160749d-7fe2-43f9-a90e-2a702762b46a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:00:32.253032 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5160749d-7fe2-43f9-a90e-2a702762b46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:32.361851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160749d-7fe2-43f9-a90e-2a702762b46a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:00:32.438762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:32.442380 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846ba42-d718-4da9-8f37-9c6eef859ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:00:32.443058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846ba42-d718-4da9-8f37-9c6eef859ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:32.443472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846ba42-d718-4da9-8f37-9c6eef859ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:32.443757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846ba42-d718-4da9-8f37-9c6eef859ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:32.446398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0846ba42-d718-4da9-8f37-9c6eef859ad7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:32.567258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846ba42-d718-4da9-8f37-9c6eef859ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:00:32.568634 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846ba42-d718-4da9-8f37-9c6eef859ad7 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:00:32.587502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 664/1314] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:00:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:00:34.944243 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf10bc-dace-4cf0-97bb-b31cffc28191 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:34.944820 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf10bc-dace-4cf0-97bb-b31cffc28191 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:34.944820 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf10bc-dace-4cf0-97bb-b31cffc28191 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:34.947035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 651/1315] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:00:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:00:34.955921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f0db1-15fb-46bd-89e0-f4c671e0201a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:34.956322 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f0db1-15fb-46bd-89e0-f4c671e0201a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:34.956544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f0db1-15fb-46bd-89e0-f4c671e0201a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:35.027901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5b7f0db1-15fb-46bd-89e0-f4c671e0201a None None] MFA Rules not processed for user `fb028e92a3234945a85d5d498019b5ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:00:35.029469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5b7f0db1-15fb-46bd-89e0-f4c671e0201a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:35.089681 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:35.138193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 665/1316] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:00:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:00:35.181284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf10bc-dace-4cf0-97bb-b31cffc28191 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:00:35.182880 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7acf10bc-dace-4cf0-97bb-b31cffc28191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:35.193891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:35.312839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf10bc-dace-4cf0-97bb-b31cffc28191 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:00:35.400391 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e4acb-a770-4e3b-be68-8f2c79b9fda7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:00:35.401053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e4acb-a770-4e3b-be68-8f2c79b9fda7 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:35.401335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e4acb-a770-4e3b-be68-8f2c79b9fda7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:35.401608 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e4acb-a770-4e3b-be68-8f2c79b9fda7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:35.403341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-616e4acb-a770-4e3b-be68-8f2c79b9fda7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:35.471352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:35.521401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e4acb-a770-4e3b-be68-8f2c79b9fda7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:00:35.522906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e4acb-a770-4e3b-be68-8f2c79b9fda7 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:00:35.542683 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 652/1317] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:00:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:00:38.155484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da65996-e5c8-4e12-bd2b-d17df925fc5d None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:38.155949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da65996-e5c8-4e12-bd2b-d17df925fc5d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:38.156078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da65996-e5c8-4e12-bd2b-d17df925fc5d None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:38.157452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 666/1318] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:00:38.162989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fbe25-5a34-40f9-b26e-ba4cab04cae4 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:38.163420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fbe25-5a34-40f9-b26e-ba4cab04cae4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:38.163534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fbe25-5a34-40f9-b26e-ba4cab04cae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:38.238327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d05fbe25-5a34-40f9-b26e-ba4cab04cae4 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:00:38.239622 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d05fbe25-5a34-40f9-b26e-ba4cab04cae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:38.343948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 653/1319] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:00:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:00:38.390298 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da65996-e5c8-4e12-bd2b-d17df925fc5d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:00:38.391797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9da65996-e5c8-4e12-bd2b-d17df925fc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:38.511070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da65996-e5c8-4e12-bd2b-d17df925fc5d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:00:38.590625 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4ba79-c05b-4261-9e64-e9dfb310ef79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:00:38.591273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4ba79-c05b-4261-9e64-e9dfb310ef79 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:38.591555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4ba79-c05b-4261-9e64-e9dfb310ef79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:38.591824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4ba79-c05b-4261-9e64-e9dfb310ef79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:38.593454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2bd4ba79-c05b-4261-9e64-e9dfb310ef79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:38.601184 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:38.726259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4ba79-c05b-4261-9e64-e9dfb310ef79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:00:38.727632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4ba79-c05b-4261-9e64-e9dfb310ef79 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:00:38.746770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 667/1320] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:00:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:00:41.516087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546856e5-79cf-4cb7-957d-dff99d247af3 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:41.516635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546856e5-79cf-4cb7-957d-dff99d247af3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:41.516635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546856e5-79cf-4cb7-957d-dff99d247af3 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:41.517996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 654/1321] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:00:41.523383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a87ccf-3b35-4fd0-badf-bc6688d96882 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:41.523831 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a87ccf-3b35-4fd0-badf-bc6688d96882 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:41.523927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a87ccf-3b35-4fd0-badf-bc6688d96882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:41.599011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-20a87ccf-3b35-4fd0-badf-bc6688d96882 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:00:41.600270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-20a87ccf-3b35-4fd0-badf-bc6688d96882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:41.667533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:41.715121 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 668/1322] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:00:41.751709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546856e5-79cf-4cb7-957d-dff99d247af3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:00:41.753249 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-546856e5-79cf-4cb7-957d-dff99d247af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:41.773339 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:41.864993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546856e5-79cf-4cb7-957d-dff99d247af3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:00:41.940779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db5bb1-4e64-45c3-8fe8-321000b23350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:00:41.941429 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db5bb1-4e64-45c3-8fe8-321000b23350 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:41.941743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db5bb1-4e64-45c3-8fe8-321000b23350 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:41.941995 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db5bb1-4e64-45c3-8fe8-321000b23350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:41.943663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-50db5bb1-4e64-45c3-8fe8-321000b23350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:42.017544 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:42.057571 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db5bb1-4e64-45c3-8fe8-321000b23350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:00:42.059738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db5bb1-4e64-45c3-8fe8-321000b23350 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:00:42.078587 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 655/1323] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:00:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:00:44.129782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32561681-7656-4617-8566-527a05d3cb21 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:44.130317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32561681-7656-4617-8566-527a05d3cb21 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:44.130317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32561681-7656-4617-8566-527a05d3cb21 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:44.132137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 669/1324] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:00:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:00:44.138156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a468e15-d480-4a56-b3f5-676f93fca523 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:44.138559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a468e15-d480-4a56-b3f5-676f93fca523 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:44.138743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a468e15-d480-4a56-b3f5-676f93fca523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:44.203368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-8a468e15-d480-4a56-b3f5-676f93fca523 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:00:44.204561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8a468e15-d480-4a56-b3f5-676f93fca523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:44.304570 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 656/1325] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:00:44.353396 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64de1-49ff-4c45-8fc1-17d05aa4c861 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:44.353815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64de1-49ff-4c45-8fc1-17d05aa4c861 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:44.354221 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64de1-49ff-4c45-8fc1-17d05aa4c861 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:44.356205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 670/1326] 10.222.0.79 () {54 vars in 964 bytes} [Mon Jul 4 10:00:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:00:44.362151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e3bcf-0b49-4e37-8408-707b45fb083e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:44.362473 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e3bcf-0b49-4e37-8408-707b45fb083e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:44.362751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e3bcf-0b49-4e37-8408-707b45fb083e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:44.391992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:44.442068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b98e3bcf-0b49-4e37-8408-707b45fb083e None None] MFA Rules not processed for user `3a1f11a4b8554a98b7626a82f01888c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:00:44.443296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b98e3bcf-0b49-4e37-8408-707b45fb083e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:44.527690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 657/1327] 10.222.0.79 () {58 vars in 1061 bytes} [Mon Jul 4 10:00:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:00:44.536947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa64de1-49ff-4c45-8fc1-17d05aa4c861 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:00:44.538511 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-baa64de1-49ff-4c45-8fc1-17d05aa4c861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:44.642503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa64de1-49ff-4c45-8fc1-17d05aa4c861 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:00:44.727644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be180376-2fa1-44e2-a042-db749523450d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:00:44.728299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be180376-2fa1-44e2-a042-db749523450d service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:44.728572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be180376-2fa1-44e2-a042-db749523450d service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:44.728844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be180376-2fa1-44e2-a042-db749523450d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:44.730514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-be180376-2fa1-44e2-a042-db749523450d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:44.747377 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:44.843956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be180376-2fa1-44e2-a042-db749523450d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:00:44.845869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be180376-2fa1-44e2-a042-db749523450d service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:00:44.864648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 671/1328] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:00:47.217703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a65d4-4910-40f3-b91c-684ab49d4604 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:47.218233 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a65d4-4910-40f3-b91c-684ab49d4604 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:47.218326 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a65d4-4910-40f3-b91c-684ab49d4604 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:47.219679 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 658/1329] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:00:47.225465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735aa23e-eb13-4d0e-998f-0e4661d5926e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:47.225932 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735aa23e-eb13-4d0e-998f-0e4661d5926e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:47.226096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735aa23e-eb13-4d0e-998f-0e4661d5926e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:47.308966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-735aa23e-eb13-4d0e-998f-0e4661d5926e None None] MFA Rules not processed for user `fb028e92a3234945a85d5d498019b5ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:00:47.310389 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-735aa23e-eb13-4d0e-998f-0e4661d5926e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:47.378535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:47.413169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 672/1330] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:00:47 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:00:47.453111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a65d4-4910-40f3-b91c-684ab49d4604 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:00:47.454871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e62a65d4-4910-40f3-b91c-684ab49d4604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:47.562456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a65d4-4910-40f3-b91c-684ab49d4604 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:00:47.583971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:47.643647 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587df42b-90af-49cd-b81f-5944ccd88621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:00:47.644297 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587df42b-90af-49cd-b81f-5944ccd88621 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:47.644582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587df42b-90af-49cd-b81f-5944ccd88621 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:47.644856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587df42b-90af-49cd-b81f-5944ccd88621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:47.646533 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-587df42b-90af-49cd-b81f-5944ccd88621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:47.762756 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587df42b-90af-49cd-b81f-5944ccd88621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:00:47.764229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587df42b-90af-49cd-b81f-5944ccd88621 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:00:47.784468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 659/1331] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:00:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:00:50.224238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0adc9-2986-4053-9126-3ccbc217de1e None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:50.224824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0adc9-2986-4053-9126-3ccbc217de1e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:50.224824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0adc9-2986-4053-9126-3ccbc217de1e None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:50.226203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 673/1332] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:00:50.232603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b25ec-a2ab-4573-90ac-e4fe9c003672 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:50.233074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b25ec-a2ab-4573-90ac-e4fe9c003672 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:50.233166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b25ec-a2ab-4573-90ac-e4fe9c003672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:50.307605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:50.310649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d98b25ec-a2ab-4573-90ac-e4fe9c003672 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:00:50.312308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d98b25ec-a2ab-4573-90ac-e4fe9c003672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:50.417108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 660/1333] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:00:50.453459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0adc9-2986-4053-9126-3ccbc217de1e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:00:50.455094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c7a0adc9-2986-4053-9126-3ccbc217de1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:50.574645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0adc9-2986-4053-9126-3ccbc217de1e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:00:50.654763 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3ff20-c002-4951-b3b1-0068729601e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:00:50.655374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3ff20-c002-4951-b3b1-0068729601e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:50.655658 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3ff20-c002-4951-b3b1-0068729601e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:50.655937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3ff20-c002-4951-b3b1-0068729601e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:50.657610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-97b3ff20-c002-4951-b3b1-0068729601e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:50.686979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:50.792211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3ff20-c002-4951-b3b1-0068729601e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:00:50.793712 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3ff20-c002-4951-b3b1-0068729601e7 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:00:50.813383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 674/1334] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:00:52.925512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b8dcb-4c50-422c-bdbb-1c72446adc2d None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:52.926170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b8dcb-4c50-422c-bdbb-1c72446adc2d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:52.926170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b8dcb-4c50-422c-bdbb-1c72446adc2d None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:52.927509 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 661/1335] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:00:52.934275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285bd91-b559-4d04-95b4-5c23a9d5be95 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:52.934707 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285bd91-b559-4d04-95b4-5c23a9d5be95 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:52.934810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285bd91-b559-4d04-95b4-5c23a9d5be95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:53.032386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f285bd91-b559-4d04-95b4-5c23a9d5be95 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:00:53.033652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f285bd91-b559-4d04-95b4-5c23a9d5be95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:53.114417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:53.138539 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 675/1336] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:00:53.182458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b8dcb-4c50-422c-bdbb-1c72446adc2d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:00:53.184894 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a81b8dcb-4c50-422c-bdbb-1c72446adc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:53.310916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b8dcb-4c50-422c-bdbb-1c72446adc2d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:00:53.343371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:53.394849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143fdb6-35b4-4c3b-b5c6-bebf023ad2e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:00:53.395506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143fdb6-35b4-4c3b-b5c6-bebf023ad2e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:53.395785 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143fdb6-35b4-4c3b-b5c6-bebf023ad2e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:53.396058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143fdb6-35b4-4c3b-b5c6-bebf023ad2e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:53.397741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2143fdb6-35b4-4c3b-b5c6-bebf023ad2e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:53.517293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143fdb6-35b4-4c3b-b5c6-bebf023ad2e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:00:53.518703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143fdb6-35b4-4c3b-b5c6-bebf023ad2e4 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:00:53.537753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 662/1337] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:00:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:00:55.946391 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396669f-e79b-4558-a433-41ce64208197 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:55.947299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396669f-e79b-4558-a433-41ce64208197 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:55.947299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396669f-e79b-4558-a433-41ce64208197 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:55.950568 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 676/1338] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:00:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:00:55.959416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f987ef0-6df3-442f-b2da-1c60bfe829f6 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:55.959846 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f987ef0-6df3-442f-b2da-1c60bfe829f6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:55.959996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f987ef0-6df3-442f-b2da-1c60bfe829f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:56.037984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7f987ef0-6df3-442f-b2da-1c60bfe829f6 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:00:56.039209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7f987ef0-6df3-442f-b2da-1c60bfe829f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:56.048429 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:56.147435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 663/1339] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:00:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:00:56.188018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396669f-e79b-4558-a433-41ce64208197 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:00:56.190525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0396669f-e79b-4558-a433-41ce64208197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:56.314732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396669f-e79b-4558-a433-41ce64208197 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:00:56.396985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbaf8d0-2c64-48fe-ba69-cf097e9cd2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:00:56.397911 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbaf8d0-2c64-48fe-ba69-cf097e9cd2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:00:56.398236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbaf8d0-2c64-48fe-ba69-cf097e9cd2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:00:56.398571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbaf8d0-2c64-48fe-ba69-cf097e9cd2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:00:56.401143 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2dbaf8d0-2c64-48fe-ba69-cf097e9cd2c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:00:56.446233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:00:56.524212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbaf8d0-2c64-48fe-ba69-cf097e9cd2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:00:56.525663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbaf8d0-2c64-48fe-ba69-cf097e9cd2c2 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:00:56.544279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 677/1340] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:00:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:00.578748 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7b4c1-4036-4329-9f2d-5791b9358360 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:00.579290 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7b4c1-4036-4329-9f2d-5791b9358360 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:00.579290 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7b4c1-4036-4329-9f2d-5791b9358360 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:00.580805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 664/1341] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:00.585917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90950e10-4f91-473d-acd2-6d7a7d5f0de0 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:00.586356 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90950e10-4f91-473d-acd2-6d7a7d5f0de0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:00.586446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90950e10-4f91-473d-acd2-6d7a7d5f0de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:00.661876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-90950e10-4f91-473d-acd2-6d7a7d5f0de0 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:00.663094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-90950e10-4f91-473d-acd2-6d7a7d5f0de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:00.760902 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:01:00.775162 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 678/1342] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:00.813960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7b4c1-4036-4329-9f2d-5791b9358360 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:00.816530 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-aeb7b4c1-4036-4329-9f2d-5791b9358360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:00.915246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7b4c1-4036-4329-9f2d-5791b9358360 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:00.918850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288c331-cc47-47e3-8b75-8674cc77a5f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:00.919586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288c331-cc47-47e3-8b75-8674cc77a5f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:00.919930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288c331-cc47-47e3-8b75-8674cc77a5f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:00.920263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288c331-cc47-47e3-8b75-8674cc77a5f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:00.929267 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288c331-cc47-47e3-8b75-8674cc77a5f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:00.930834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288c331-cc47-47e3-8b75-8674cc77a5f3 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:00.940153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 665/1343] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:03.796982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d084708-6683-4aa7-96ae-3872cb1255e7 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:03.797578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d084708-6683-4aa7-96ae-3872cb1255e7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:03.797578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d084708-6683-4aa7-96ae-3872cb1255e7 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:03.799026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 679/1344] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:03.805119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe88ae-6742-4cec-8cb3-ef3811867afe None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:03.805723 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe88ae-6742-4cec-8cb3-ef3811867afe None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:03.805832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe88ae-6742-4cec-8cb3-ef3811867afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:03.836842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-cbfe88ae-6742-4cec-8cb3-ef3811867afe None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:03.838106 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-cbfe88ae-6742-4cec-8cb3-ef3811867afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:03.870402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:01:03.925144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 666/1345] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:03.962435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d084708-6683-4aa7-96ae-3872cb1255e7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:03.974153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d084708-6683-4aa7-96ae-3872cb1255e7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:03.978243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b249526-999f-46c4-871e-564a9b475d05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:03.978930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b249526-999f-46c4-871e-564a9b475d05 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:03.979219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b249526-999f-46c4-871e-564a9b475d05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:03.979503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b249526-999f-46c4-871e-564a9b475d05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:03.989637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b249526-999f-46c4-871e-564a9b475d05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:03.991121 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b249526-999f-46c4-871e-564a9b475d05 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:04.001395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 680/1346] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:08.202207 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49317-5d98-44a1-b030-2527033cc762 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:08.202827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49317-5d98-44a1-b030-2527033cc762 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:08.202827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49317-5d98-44a1-b030-2527033cc762 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:08.204305 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 667/1347] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:08.209447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a6bc0-5483-4ac4-a442-5dca19273c07 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:08.209903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a6bc0-5483-4ac4-a442-5dca19273c07 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:08.210070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a6bc0-5483-4ac4-a442-5dca19273c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:08.228978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f21a6bc0-5483-4ac4-a442-5dca19273c07 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:08.230270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f21a6bc0-5483-4ac4-a442-5dca19273c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:08.238734 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 681/1348] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:08.276395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b49317-5d98-44a1-b030-2527033cc762 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:08.278286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-33b49317-5d98-44a1-b030-2527033cc762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:08.291966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b49317-5d98-44a1-b030-2527033cc762 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:08.295764 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a09f8-0389-46a4-968a-f298e32ff3d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:08.296389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a09f8-0389-46a4-968a-f298e32ff3d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:08.296674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a09f8-0389-46a4-968a-f298e32ff3d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:08.296946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a09f8-0389-46a4-968a-f298e32ff3d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:08.306160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a09f8-0389-46a4-968a-f298e32ff3d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:08.307614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a09f8-0389-46a4-968a-f298e32ff3d6 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:08.317014 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 668/1349] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:13.360075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4a659-c2a8-4129-8474-3c6570e0a033 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:13.360075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4a659-c2a8-4129-8474-3c6570e0a033 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:13.360526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4a659-c2a8-4129-8474-3c6570e0a033 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:13.361996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 682/1350] 10.222.0.79 () {54 vars in 921 bytes} [Mon Jul 4 10:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:13.367709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aacbbc-36e9-455b-b792-b8eb412ab974 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:13.368537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aacbbc-36e9-455b-b792-b8eb412ab974 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:13.369031 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aacbbc-36e9-455b-b792-b8eb412ab974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:13.522376 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d2aacbbc-36e9-455b-b792-b8eb412ab974 None None] MFA Rules not processed for user `5dc48318ab7944c197bf5f7673511931`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:13.523927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d2aacbbc-36e9-455b-b792-b8eb412ab974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:13.617848 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 669/1351] 10.222.0.79 () {58 vars in 1018 bytes} [Mon Jul 4 10:01:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:01:17.715379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e902b9d-ed8a-4815-aa58-5e93012566a9 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:17.715947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e902b9d-ed8a-4815-aa58-5e93012566a9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:17.715947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e902b9d-ed8a-4815-aa58-5e93012566a9 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:17.717302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 683/1352] 10.222.0.79 () {54 vars in 921 bytes} [Mon Jul 4 10:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:17.723126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b493a15-b3fa-4439-ad6c-0bd0940f410c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:17.723665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b493a15-b3fa-4439-ad6c-0bd0940f410c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:17.723665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b493a15-b3fa-4439-ad6c-0bd0940f410c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:17.742380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-3b493a15-b3fa-4439-ad6c-0bd0940f410c None None] MFA Rules not processed for user `5dc48318ab7944c197bf5f7673511931`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:17.743589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3b493a15-b3fa-4439-ad6c-0bd0940f410c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:17.755666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 670/1353] 10.222.0.79 () {58 vars in 1018 bytes} [Mon Jul 4 10:01:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:18.223843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=13 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:01:19.286686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566e7cc-d179-4d1b-b296-4e95877812cb None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:19.287309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566e7cc-d179-4d1b-b296-4e95877812cb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:19.287309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566e7cc-d179-4d1b-b296-4e95877812cb None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:19.288609 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 684/1354] 10.222.0.79 () {54 vars in 921 bytes} [Mon Jul 4 10:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:19.293942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d4dea-d9c9-4034-902e-e611315eeaff None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:19.294398 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d4dea-d9c9-4034-902e-e611315eeaff None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:19.294654 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d4dea-d9c9-4034-902e-e611315eeaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:19.313311 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-523d4dea-d9c9-4034-902e-e611315eeaff None None] MFA Rules not processed for user `5dc48318ab7944c197bf5f7673511931`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:19.314563 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-523d4dea-d9c9-4034-902e-e611315eeaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:19.324179 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 671/1355] 10.222.0.79 () {58 vars in 1018 bytes} [Mon Jul 4 10:01:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:01:22.907059 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f2290-195d-4615-994b-1631c5b8fce7 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:22.908795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f2290-195d-4615-994b-1631c5b8fce7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:22.909249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f2290-195d-4615-994b-1631c5b8fce7 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:22.911302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 685/1356] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:22.917665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ae358-53d8-4bdb-8d0d-f64f4f05a4bc None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:22.917665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ae358-53d8-4bdb-8d0d-f64f4f05a4bc None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:22.917665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ae358-53d8-4bdb-8d0d-f64f4f05a4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:22.942715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-214ae358-53d8-4bdb-8d0d-f64f4f05a4bc None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:22.944165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-214ae358-53d8-4bdb-8d0d-f64f4f05a4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:22.958167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 672/1357] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:01:23.016312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1753296-6ed4-49b0-bdb8-3c8553a9cd9d None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:23.016597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1753296-6ed4-49b0-bdb8-3c8553a9cd9d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:23.017057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1753296-6ed4-49b0-bdb8-3c8553a9cd9d None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:23.018455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 686/1358] 10.222.0.79 () {54 vars in 961 bytes} [Mon Jul 4 10:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:23.026289 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f197234-bbe3-49cf-9a7c-c77615835169 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:23.026703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f197234-bbe3-49cf-9a7c-c77615835169 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:23.027088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f197234-bbe3-49cf-9a7c-c77615835169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:23.073701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:01:23.082026 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4f197234-bbe3-49cf-9a7c-c77615835169 None None] MFA Rules not processed for user `ffc2409cb7fc4373a74ddfb48d501d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:23.083596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4f197234-bbe3-49cf-9a7c-c77615835169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:23.188714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 673/1359] 10.222.0.79 () {58 vars in 1058 bytes} [Mon Jul 4 10:01:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:23.196683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1753296-6ed4-49b0-bdb8-3c8553a9cd9d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:23.213238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1753296-6ed4-49b0-bdb8-3c8553a9cd9d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:23.220503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da188cc7-4347-4856-8939-48edf55e5eca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:23.221468 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da188cc7-4347-4856-8939-48edf55e5eca service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:23.221982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da188cc7-4347-4856-8939-48edf55e5eca service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:23.222423 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da188cc7-4347-4856-8939-48edf55e5eca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:23.232045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da188cc7-4347-4856-8939-48edf55e5eca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:23.233720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da188cc7-4347-4856-8939-48edf55e5eca service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:23.244749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 687/1360] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:01:25.515441 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f6589-560c-4791-b3f0-4c8f2b106afe None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:25.516024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f6589-560c-4791-b3f0-4c8f2b106afe None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:25.516024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f6589-560c-4791-b3f0-4c8f2b106afe None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:25.517557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 674/1361] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:25.523060 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcfc65-fe51-47c7-b233-75f73720fc32 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:25.523482 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcfc65-fe51-47c7-b233-75f73720fc32 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:25.523622 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcfc65-fe51-47c7-b233-75f73720fc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:25.543106 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d9dcfc65-fe51-47c7-b233-75f73720fc32 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:25.544373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d9dcfc65-fe51-47c7-b233-75f73720fc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:25.553263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 688/1362] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:25.598103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f6589-560c-4791-b3f0-4c8f2b106afe None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:25.614152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f6589-560c-4791-b3f0-4c8f2b106afe None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:25.620504 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5124d-705f-41ea-8c4d-33d0c5cbe521 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:25.621262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5124d-705f-41ea-8c4d-33d0c5cbe521 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:25.621635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5124d-705f-41ea-8c4d-33d0c5cbe521 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:25.622002 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5124d-705f-41ea-8c4d-33d0c5cbe521 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:25.631281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5124d-705f-41ea-8c4d-33d0c5cbe521 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:25.632990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5124d-705f-41ea-8c4d-33d0c5cbe521 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:25.643956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 675/1363] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:27.961009 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc80b8a-7b0e-49c8-8354-336db6000884 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:27.961579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc80b8a-7b0e-49c8-8354-336db6000884 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:27.961579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc80b8a-7b0e-49c8-8354-336db6000884 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:27.963241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 689/1364] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:27.968522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c309f-f1af-47f8-824b-6206150b76ff None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:27.969320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c309f-f1af-47f8-824b-6206150b76ff None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:27.969425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c309f-f1af-47f8-824b-6206150b76ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:27.994471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d60c309f-f1af-47f8-824b-6206150b76ff None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:27.996037 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d60c309f-f1af-47f8-824b-6206150b76ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:28.006394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 676/1365] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:28.051716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd624485-4a89-4fc9-a314-8da64d3b8750 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:28.052006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd624485-4a89-4fc9-a314-8da64d3b8750 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:28.052313 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd624485-4a89-4fc9-a314-8da64d3b8750 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:28.053826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 690/1366] 10.222.0.79 () {54 vars in 961 bytes} [Mon Jul 4 10:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:28.059387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce6f2-fb57-4d5f-acb1-8e51f0898b66 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:28.059820 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce6f2-fb57-4d5f-acb1-8e51f0898b66 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:28.060100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce6f2-fb57-4d5f-acb1-8e51f0898b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:28.082518 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-893ce6f2-fb57-4d5f-acb1-8e51f0898b66 None None] MFA Rules not processed for user `ffc2409cb7fc4373a74ddfb48d501d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:28.083724 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-893ce6f2-fb57-4d5f-acb1-8e51f0898b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:28.096113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 677/1367] 10.222.0.79 () {58 vars in 1058 bytes} [Mon Jul 4 10:01:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:28.104424 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd624485-4a89-4fc9-a314-8da64d3b8750 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:28.120012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd624485-4a89-4fc9-a314-8da64d3b8750 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:28.125410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a919b-4f12-4f1a-adf4-5991de515c50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:28.126352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a919b-4f12-4f1a-adf4-5991de515c50 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:28.126814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a919b-4f12-4f1a-adf4-5991de515c50 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:28.127093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a919b-4f12-4f1a-adf4-5991de515c50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:28.136235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a919b-4f12-4f1a-adf4-5991de515c50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:28.137896 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a919b-4f12-4f1a-adf4-5991de515c50 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:28.147749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 691/1368] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:01:30.259676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dbc02-0535-4f66-8a5c-44b6f69b592b None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:30.260775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dbc02-0535-4f66-8a5c-44b6f69b592b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:30.261274 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dbc02-0535-4f66-8a5c-44b6f69b592b None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:30.263045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 678/1369] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:30.269308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fee63-f8e4-49b0-93fd-a7dee448dc17 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:30.270049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fee63-f8e4-49b0-93fd-a7dee448dc17 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:30.270049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fee63-f8e4-49b0-93fd-a7dee448dc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:30.290001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-fb5fee63-f8e4-49b0-93fd-a7dee448dc17 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:30.291652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fb5fee63-f8e4-49b0-93fd-a7dee448dc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:30.301646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 692/1370] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:30.346248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dbc02-0535-4f66-8a5c-44b6f69b592b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:30.379989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dbc02-0535-4f66-8a5c-44b6f69b592b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:30.387509 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258a758-ecf8-4cf9-806f-7ece9c3a4613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:30.388510 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258a758-ecf8-4cf9-806f-7ece9c3a4613 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:30.388859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258a758-ecf8-4cf9-806f-7ece9c3a4613 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:30.389368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258a758-ecf8-4cf9-806f-7ece9c3a4613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:30.399759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258a758-ecf8-4cf9-806f-7ece9c3a4613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:30.401443 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258a758-ecf8-4cf9-806f-7ece9c3a4613 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:30.411927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 679/1371] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:30.965473 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de707daf-4f48-48ae-b5b3-aec5bfe27306 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:30.966076 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de707daf-4f48-48ae-b5b3-aec5bfe27306 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:30.966504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de707daf-4f48-48ae-b5b3-aec5bfe27306 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:30.968046 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 693/1372] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 10:01:30.973884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f42ef-3ec8-483c-af1f-4d497044b086 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:30.974346 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f42ef-3ec8-483c-af1f-4d497044b086 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:30.974798 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f42ef-3ec8-483c-af1f-4d497044b086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:30.993381 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-db3f42ef-3ec8-483c-af1f-4d497044b086 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:30.994658 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-db3f42ef-3ec8-483c-af1f-4d497044b086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:31.003349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 680/1373] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:31.343387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d555b61-5124-49f5-ad70-893438062278 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:31.343387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d555b61-5124-49f5-ad70-893438062278 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:31.343387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d555b61-5124-49f5-ad70-893438062278 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:31.345907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 694/1374] 10.222.0.79 () {54 vars in 963 bytes} [Mon Jul 4 10:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:31.350866 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7014cd0-8dcb-476d-8520-f1c87a52b8c2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:31.351615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7014cd0-8dcb-476d-8520-f1c87a52b8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:31.351615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7014cd0-8dcb-476d-8520-f1c87a52b8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:31.409183 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b7014cd0-8dcb-476d-8520-f1c87a52b8c2 None None] MFA Rules not processed for user `c5a4af83902d4370a832ed3ea3861b56`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:31.411232 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b7014cd0-8dcb-476d-8520-f1c87a52b8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:31.480920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 681/1375] 10.222.0.79 () {58 vars in 1060 bytes} [Mon Jul 4 10:01:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:01:31.489027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d555b61-5124-49f5-ad70-893438062278 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:31.500493 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d555b61-5124-49f5-ad70-893438062278 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:31.504543 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ab5f4-b124-4c83-8170-cb00663beb69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:31.505195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ab5f4-b124-4c83-8170-cb00663beb69 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:31.505478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ab5f4-b124-4c83-8170-cb00663beb69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:31.505813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ab5f4-b124-4c83-8170-cb00663beb69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:31.516199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ab5f4-b124-4c83-8170-cb00663beb69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:31.517673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ab5f4-b124-4c83-8170-cb00663beb69 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:31.528387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 695/1376] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:32.512566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcef639-56fa-454a-9d19-953bc7314c7d None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:32.513088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcef639-56fa-454a-9d19-953bc7314c7d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:32.513088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcef639-56fa-454a-9d19-953bc7314c7d None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:32.514521 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 682/1377] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:32.520182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac1f94-67a0-4bfd-bfa8-92ca3717cb60 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:32.520607 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac1f94-67a0-4bfd-bfa8-92ca3717cb60 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:32.520747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac1f94-67a0-4bfd-bfa8-92ca3717cb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:32.540637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-84ac1f94-67a0-4bfd-bfa8-92ca3717cb60 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:32.542607 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-84ac1f94-67a0-4bfd-bfa8-92ca3717cb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:32.551502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 696/1378] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:32.595665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcef639-56fa-454a-9d19-953bc7314c7d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:32.611178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcef639-56fa-454a-9d19-953bc7314c7d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:32.617375 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3295e62-326e-4ac5-a158-e2bb0a69ebfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:32.618054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3295e62-326e-4ac5-a158-e2bb0a69ebfe service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:32.618340 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3295e62-326e-4ac5-a158-e2bb0a69ebfe service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:32.618708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3295e62-326e-4ac5-a158-e2bb0a69ebfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:32.628034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3295e62-326e-4ac5-a158-e2bb0a69ebfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:32.629997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3295e62-326e-4ac5-a158-e2bb0a69ebfe service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:32.633921 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:01:32.641919 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 683/1379] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:34.703650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396fb45-4c72-488b-8fee-1e897ca31f8f None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:34.704172 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396fb45-4c72-488b-8fee-1e897ca31f8f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:34.704269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396fb45-4c72-488b-8fee-1e897ca31f8f None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:34.705714 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 697/1380] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:34.711172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c07076-5969-4035-9e85-54c88f16e187 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:34.711603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c07076-5969-4035-9e85-54c88f16e187 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:34.711954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c07076-5969-4035-9e85-54c88f16e187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:34.732130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-76c07076-5969-4035-9e85-54c88f16e187 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:34.733484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-76c07076-5969-4035-9e85-54c88f16e187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:34.741883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 684/1381] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:34.786043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396fb45-4c72-488b-8fee-1e897ca31f8f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:34.800867 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396fb45-4c72-488b-8fee-1e897ca31f8f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:34.806635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afdc9d-6859-49a7-a49b-98292b151818 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:34.807283 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afdc9d-6859-49a7-a49b-98292b151818 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:34.807573 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afdc9d-6859-49a7-a49b-98292b151818 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:34.807879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afdc9d-6859-49a7-a49b-98292b151818 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:34.816188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afdc9d-6859-49a7-a49b-98292b151818 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:34.817806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afdc9d-6859-49a7-a49b-98292b151818 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:34.827403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 698/1382] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:36.920470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c835f-5bf5-41b0-90df-94d71f32a68b None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:36.921054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c835f-5bf5-41b0-90df-94d71f32a68b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:36.921054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c835f-5bf5-41b0-90df-94d71f32a68b None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:36.922426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 685/1383] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:36.927895 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63753f8d-942e-4f87-8a95-37c93a44c883 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:36.928318 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63753f8d-942e-4f87-8a95-37c93a44c883 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:36.928431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63753f8d-942e-4f87-8a95-37c93a44c883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:36.947805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-63753f8d-942e-4f87-8a95-37c93a44c883 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:36.948922 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-63753f8d-942e-4f87-8a95-37c93a44c883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:36.957097 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 699/1384] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:37.004272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c835f-5bf5-41b0-90df-94d71f32a68b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:37.021136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c835f-5bf5-41b0-90df-94d71f32a68b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:37.027890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3dd96-9745-4354-b8f4-346cee3bcf84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:37.028921 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3dd96-9745-4354-b8f4-346cee3bcf84 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:37.029316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3dd96-9745-4354-b8f4-346cee3bcf84 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:37.029743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3dd96-9745-4354-b8f4-346cee3bcf84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:37.040670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3dd96-9745-4354-b8f4-346cee3bcf84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:37.042504 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3dd96-9745-4354-b8f4-346cee3bcf84 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:37.055554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 686/1385] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:39.139608 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06313b33-54ea-48d8-a890-6b32197fd12a None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:39.140219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06313b33-54ea-48d8-a890-6b32197fd12a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:39.140219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06313b33-54ea-48d8-a890-6b32197fd12a None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:39.141585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 700/1386] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 10:01:39.147621 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445b3db-c9d5-4619-bb20-d3f2da91f447 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:39.148148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445b3db-c9d5-4619-bb20-d3f2da91f447 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:39.148148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445b3db-c9d5-4619-bb20-d3f2da91f447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:39.167517 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d445b3db-c9d5-4619-bb20-d3f2da91f447 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:39.168739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d445b3db-c9d5-4619-bb20-d3f2da91f447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:39.177200 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 687/1387] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:39.222567 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06313b33-54ea-48d8-a890-6b32197fd12a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:39.237271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06313b33-54ea-48d8-a890-6b32197fd12a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:39.242650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200dac7f-539b-4aaf-9ef2-622ad34c2384 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:39.243302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dac7f-539b-4aaf-9ef2-622ad34c2384 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:39.243619 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dac7f-539b-4aaf-9ef2-622ad34c2384 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:39.243898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dac7f-539b-4aaf-9ef2-622ad34c2384 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:39.252424 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200dac7f-539b-4aaf-9ef2-622ad34c2384 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:39.254063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200dac7f-539b-4aaf-9ef2-622ad34c2384 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:39.263664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 701/1388] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:41.331536 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89a95f-1549-4991-8a2e-688d582010ac None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:41.332088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89a95f-1549-4991-8a2e-688d582010ac None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:41.332088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89a95f-1549-4991-8a2e-688d582010ac None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:41.333748 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 688/1389] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:41.340171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb1a43-f370-469d-8bde-15937624003d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:41.340629 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb1a43-f370-469d-8bde-15937624003d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:41.340753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb1a43-f370-469d-8bde-15937624003d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:41.360999 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-aadb1a43-f370-469d-8bde-15937624003d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:41.362279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-aadb1a43-f370-469d-8bde-15937624003d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:41.371443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 702/1390] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:41.417228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89a95f-1549-4991-8a2e-688d582010ac None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:41.433045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89a95f-1549-4991-8a2e-688d582010ac None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:41.439133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e571410-83f9-4e6a-8fb0-b777217f363d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:41.439785 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e571410-83f9-4e6a-8fb0-b777217f363d service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:41.440071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e571410-83f9-4e6a-8fb0-b777217f363d service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:41.440348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e571410-83f9-4e6a-8fb0-b777217f363d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:41.457463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e571410-83f9-4e6a-8fb0-b777217f363d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:41.459147 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e571410-83f9-4e6a-8fb0-b777217f363d service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:41.475606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 689/1391] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:43.530884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8abe-18d2-4ac8-9da5-11defc3c3dea None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:43.531445 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8abe-18d2-4ac8-9da5-11defc3c3dea None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:43.531613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8abe-18d2-4ac8-9da5-11defc3c3dea None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:43.533379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 703/1392] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:43.540577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950f0de-61e7-43c9-aa89-9b575ab22f9a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:43.541000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950f0de-61e7-43c9-aa89-9b575ab22f9a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:43.541301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950f0de-61e7-43c9-aa89-9b575ab22f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:43.564821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-8950f0de-61e7-43c9-aa89-9b575ab22f9a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:43.566477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8950f0de-61e7-43c9-aa89-9b575ab22f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:43.577932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 690/1393] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:43.633933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d8abe-18d2-4ac8-9da5-11defc3c3dea None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:43.649946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d8abe-18d2-4ac8-9da5-11defc3c3dea None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:43.655913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94811c8a-7a4a-4514-ba3c-e2661ac1bafa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:43.656570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94811c8a-7a4a-4514-ba3c-e2661ac1bafa service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:43.656856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94811c8a-7a4a-4514-ba3c-e2661ac1bafa service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:43.657135 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94811c8a-7a4a-4514-ba3c-e2661ac1bafa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:43.666120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94811c8a-7a4a-4514-ba3c-e2661ac1bafa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:43.667819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94811c8a-7a4a-4514-ba3c-e2661ac1bafa service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:43.677781 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 704/1394] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:45.715908 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42011e-e41a-47f2-b200-ecfc411bb0ee None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:45.716484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42011e-e41a-47f2-b200-ecfc411bb0ee None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:45.716484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42011e-e41a-47f2-b200-ecfc411bb0ee None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:45.718042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 691/1395] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:45.723482 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960aff57-8c44-450a-9ada-9ed88902629e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:45.723915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960aff57-8c44-450a-9ada-9ed88902629e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:45.724005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960aff57-8c44-450a-9ada-9ed88902629e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:45.742261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-960aff57-8c44-450a-9ada-9ed88902629e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:45.743465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-960aff57-8c44-450a-9ada-9ed88902629e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:45.752725 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 705/1396] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:45.797021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42011e-e41a-47f2-b200-ecfc411bb0ee None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:45.812901 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42011e-e41a-47f2-b200-ecfc411bb0ee None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:45.818550 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18ce08-9239-4263-bb07-0c0ad94f809f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:45.819213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18ce08-9239-4263-bb07-0c0ad94f809f service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:45.819498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18ce08-9239-4263-bb07-0c0ad94f809f service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:45.819782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18ce08-9239-4263-bb07-0c0ad94f809f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:45.829808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18ce08-9239-4263-bb07-0c0ad94f809f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:45.831420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18ce08-9239-4263-bb07-0c0ad94f809f service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:45.841085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 692/1397] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:46.571012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798fa4a-970f-4159-92b2-efc722cff228 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:46.571254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798fa4a-970f-4159-92b2-efc722cff228 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:46.571602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798fa4a-970f-4159-92b2-efc722cff228 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:46.573107 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 706/1398] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:46.578395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966ddbc-a689-4974-bba0-ff6fcb3231b5 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:46.578649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966ddbc-a689-4974-bba0-ff6fcb3231b5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:46.578965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966ddbc-a689-4974-bba0-ff6fcb3231b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:46.598383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5966ddbc-a689-4974-bba0-ff6fcb3231b5 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:46.599633 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5966ddbc-a689-4974-bba0-ff6fcb3231b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:46.608474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 693/1399] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:46.663600 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5f404-b616-4ab7-8f16-b76a4cc31f10 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:46.663884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5f404-b616-4ab7-8f16-b76a4cc31f10 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:46.664161 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5f404-b616-4ab7-8f16-b76a4cc31f10 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:46.665676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 707/1400] 10.222.0.79 () {54 vars in 962 bytes} [Mon Jul 4 10:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:46.670829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d1d09-b11e-48b2-953d-f718b9adede1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:46.671124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d1d09-b11e-48b2-953d-f718b9adede1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:46.671405 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d1d09-b11e-48b2-953d-f718b9adede1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:46.720412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e34d1d09-b11e-48b2-953d-f718b9adede1 None None] MFA Rules not processed for user `fef685a17c804e2bbfea1e07cfdf5cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:46.722076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e34d1d09-b11e-48b2-953d-f718b9adede1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:46.782444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 694/1401] 10.222.0.79 () {58 vars in 1059 bytes} [Mon Jul 4 10:01:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:46.789871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5f404-b616-4ab7-8f16-b76a4cc31f10 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:46.801369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5f404-b616-4ab7-8f16-b76a4cc31f10 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:46.805298 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38651940-cf14-4b16-adec-fdda1b301157 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef685a17c804e2bbfea1e07cfdf5cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:46.806046 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651940-cf14-4b16-adec-fdda1b301157 service glance] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:46.806334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651940-cf14-4b16-adec-fdda1b301157 service glance] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:46.806612 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651940-cf14-4b16-adec-fdda1b301157 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:46.816406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38651940-cf14-4b16-adec-fdda1b301157 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:46.817891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38651940-cf14-4b16-adec-fdda1b301157 service glance] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:46.828093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 708/1402] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:47.562383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360aefb-af02-4b8c-878d-b8f2a2b2865f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:47.562685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360aefb-af02-4b8c-878d-b8f2a2b2865f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:47.562968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360aefb-af02-4b8c-878d-b8f2a2b2865f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:47.614427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-3360aefb-af02-4b8c-878d-b8f2a2b2865f None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:47.615594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3360aefb-af02-4b8c-878d-b8f2a2b2865f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:47.656382 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:01:47.680805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 695/1403] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:01:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:47.721546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec69417-5f12-43c8-99b0-0ac85bf3a146 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:47.721935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec69417-5f12-43c8-99b0-0ac85bf3a146 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:47.722276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec69417-5f12-43c8-99b0-0ac85bf3a146 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:47.723712 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 709/1404] 10.222.0.79 () {54 vars in 942 bytes} [Mon Jul 4 10:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:47.730146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb32626-317f-42dc-815b-8cae189394d3 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:47.730615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb32626-317f-42dc-815b-8cae189394d3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:47.730713 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb32626-317f-42dc-815b-8cae189394d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:47.779953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5cb32626-317f-42dc-815b-8cae189394d3 None None] MFA Rules not processed for user `7ec37893ec8c480990c17cb03cb0ca19`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:47.781150 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5cb32626-317f-42dc-815b-8cae189394d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:47.854537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 696/1405] 10.222.0.79 () {58 vars in 1039 bytes} [Mon Jul 4 10:01:47 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:01:47.863660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec69417-5f12-43c8-99b0-0ac85bf3a146 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:47.877660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec69417-5f12-43c8-99b0-0ac85bf3a146 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:47.883214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2ac55-bc1e-4d3b-81fe-344bdd6cda48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:47.883877 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2ac55-bc1e-4d3b-81fe-344bdd6cda48 service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:47.884196 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2ac55-bc1e-4d3b-81fe-344bdd6cda48 service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:47.884474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2ac55-bc1e-4d3b-81fe-344bdd6cda48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:47.893349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2ac55-bc1e-4d3b-81fe-344bdd6cda48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:47.895162 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2ac55-bc1e-4d3b-81fe-344bdd6cda48 service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:47.903442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 710/1406] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:01:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 10:01:47.926795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb44153-f8be-4b9d-a4e6-5a510f8f3cda None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:47.927088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb44153-f8be-4b9d-a4e6-5a510f8f3cda None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:47.927367 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb44153-f8be-4b9d-a4e6-5a510f8f3cda None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:47.928746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 697/1407] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:01:47.935136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88cab8-b548-44c1-9546-075f38eb6684 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:47.935434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88cab8-b548-44c1-9546-075f38eb6684 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:47.935721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88cab8-b548-44c1-9546-075f38eb6684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:47.956449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0f88cab8-b548-44c1-9546-075f38eb6684 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:01:47.957710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0f88cab8-b548-44c1-9546-075f38eb6684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:01:47.967334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 711/1408] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:01:48.014644 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb44153-f8be-4b9d-a4e6-5a510f8f3cda None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:01:48.030690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb44153-f8be-4b9d-a4e6-5a510f8f3cda None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:01:48.038104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1e97e-d9fc-438a-9f27-6a94d0645449 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:01:48.038760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1e97e-d9fc-438a-9f27-6a94d0645449 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:01:48.039034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1e97e-d9fc-438a-9f27-6a94d0645449 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:01:48.039311 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1e97e-d9fc-438a-9f27-6a94d0645449 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:01:48.048872 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1e97e-d9fc-438a-9f27-6a94d0645449 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:01:48.050530 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1e97e-d9fc-438a-9f27-6a94d0645449 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:01:48.060676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 698/1409] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:01:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:01:57.950806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=69 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:01:58.055881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=59 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:02:10.157018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494379a3-766d-4ca4-ab6b-32d1bd5907b2 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:10.157551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494379a3-766d-4ca4-ab6b-32d1bd5907b2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:10.157551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494379a3-766d-4ca4-ab6b-32d1bd5907b2 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:10.158990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 712/1410] 10.222.0.79 () {54 vars in 918 bytes} [Mon Jul 4 10:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:10.164338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db294d0-6870-47a8-9dd7-7ef7563182c7 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:10.164896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db294d0-6870-47a8-9dd7-7ef7563182c7 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:10.164896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db294d0-6870-47a8-9dd7-7ef7563182c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:10.184056 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-3db294d0-6870-47a8-9dd7-7ef7563182c7 None None] MFA Rules not processed for user `5dc48318ab7944c197bf5f7673511931`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:10.185250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3db294d0-6870-47a8-9dd7-7ef7563182c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:10.195867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 699/1411] 10.222.0.79 () {58 vars in 1015 bytes} [Mon Jul 4 10:02:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:10.213059 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070720d7-00b3-406e-9395-703a86b3cf62 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:10.213352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070720d7-00b3-406e-9395-703a86b3cf62 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:10.213648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070720d7-00b3-406e-9395-703a86b3cf62 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:10.215031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 713/1412] 10.222.0.79 () {54 vars in 918 bytes} [Mon Jul 4 10:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:10.220104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9d264-83dd-42cd-9399-a5149ce44413 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:10.220387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9d264-83dd-42cd-9399-a5149ce44413 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:10.220666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9d264-83dd-42cd-9399-a5149ce44413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:10.238736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b0c9d264-83dd-42cd-9399-a5149ce44413 None None] MFA Rules not processed for user `ffc2409cb7fc4373a74ddfb48d501d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:10.239904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b0c9d264-83dd-42cd-9399-a5149ce44413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:10.257065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 700/1413] 10.222.0.79 () {58 vars in 1015 bytes} [Mon Jul 4 10:02:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:02:12.776244 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746a005-a437-476e-adc8-4dd1236ab0e2 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:12.777108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746a005-a437-476e-adc8-4dd1236ab0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:12.777301 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746a005-a437-476e-adc8-4dd1236ab0e2 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:12.779377 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:02:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:12.784919 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b16d7-0ca2-47c5-8ae1-e6fdde7d6a6b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:12.785698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b16d7-0ca2-47c5-8ae1-e6fdde7d6a6b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:12.785698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b16d7-0ca2-47c5-8ae1-e6fdde7d6a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:12.805068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7e7b16d7-0ca2-47c5-8ae1-e6fdde7d6a6b None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:12.806369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7e7b16d7-0ca2-47c5-8ae1-e6fdde7d6a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:12.815469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:12.870525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746a005-a437-476e-adc8-4dd1236ab0e2 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:12.888080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746a005-a437-476e-adc8-4dd1236ab0e2 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:12.897657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483210b9-92de-42a8-9d61-805d5a3ec743 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:12.898768 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483210b9-92de-42a8-9d61-805d5a3ec743 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:12.899246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483210b9-92de-42a8-9d61-805d5a3ec743 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:12.899612 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483210b9-92de-42a8-9d61-805d5a3ec743 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:12.910419 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483210b9-92de-42a8-9d61-805d5a3ec743 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:12.912039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483210b9-92de-42a8-9d61-805d5a3ec743 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:12.925510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 715/1416] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:13.864260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e769e3-43c8-42d7-9ee1-838c2165fb5e None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:13.865198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e769e3-43c8-42d7-9ee1-838c2165fb5e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:13.865198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e769e3-43c8-42d7-9ee1-838c2165fb5e None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:13.867455 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 10:02:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:13.873675 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced9f12-76ab-42a1-ad4a-2feef2daf0c5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:13.874143 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced9f12-76ab-42a1-ad4a-2feef2daf0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:13.874352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced9f12-76ab-42a1-ad4a-2feef2daf0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:13.898443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1ced9f12-76ab-42a1-ad4a-2feef2daf0c5 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:13.900101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1ced9f12-76ab-42a1-ad4a-2feef2daf0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:13.909215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 10:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:14.060497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e769e3-43c8-42d7-9ee1-838c2165fb5e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:14.077044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e769e3-43c8-42d7-9ee1-838c2165fb5e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:14.083888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e998d-b502-46c9-bb3f-6b968a006313 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:14.084540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e998d-b502-46c9-bb3f-6b968a006313 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:14.084822 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e998d-b502-46c9-bb3f-6b968a006313 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:14.085104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e998d-b502-46c9-bb3f-6b968a006313 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:14.094266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e998d-b502-46c9-bb3f-6b968a006313 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:14.095858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e998d-b502-46c9-bb3f-6b968a006313 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:14.105283 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 703/1419] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:14.998711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80366615-8526-439a-9d8f-b4dee883a25c None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:14.999280 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80366615-8526-439a-9d8f-b4dee883a25c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:14.999280 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80366615-8526-439a-9d8f-b4dee883a25c None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:15.000537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 10:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:15.006729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824fb24-7577-4951-a21c-68be325bb655 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:15.007431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824fb24-7577-4951-a21c-68be325bb655 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:15.007594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824fb24-7577-4951-a21c-68be325bb655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:15.029507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5824fb24-7577-4951-a21c-68be325bb655 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:15.030745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5824fb24-7577-4951-a21c-68be325bb655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:15.040148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 10:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:15.200660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80366615-8526-439a-9d8f-b4dee883a25c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:15.222125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80366615-8526-439a-9d8f-b4dee883a25c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:15.229063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b996b-e86e-48e2-a1b8-54e02ee4c673 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:15.229733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b996b-e86e-48e2-a1b8-54e02ee4c673 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:15.230021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b996b-e86e-48e2-a1b8-54e02ee4c673 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:15.230296 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b996b-e86e-48e2-a1b8-54e02ee4c673 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:15.238865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b996b-e86e-48e2-a1b8-54e02ee4c673 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:15.240457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b996b-e86e-48e2-a1b8-54e02ee4c673 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:15.250472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 718/1422] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:16.107610 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12b849-0cdc-4a62-b459-0ff78b9ee6e9 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:16.108389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12b849-0cdc-4a62-b459-0ff78b9ee6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:16.108389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12b849-0cdc-4a62-b459-0ff78b9ee6e9 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:16.110495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 10:02:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:16.115561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290c468-ead3-489f-a548-9ef286c9243f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:16.116003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290c468-ead3-489f-a548-9ef286c9243f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:16.116119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290c468-ead3-489f-a548-9ef286c9243f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:16.135601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d290c468-ead3-489f-a548-9ef286c9243f None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:16.136450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d290c468-ead3-489f-a548-9ef286c9243f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:16.145740 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 10:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:16.295335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12b849-0cdc-4a62-b459-0ff78b9ee6e9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:16.315721 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12b849-0cdc-4a62-b459-0ff78b9ee6e9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:16.322468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dce5a-bf19-428b-b328-cee7abc27092 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:16.323148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dce5a-bf19-428b-b328-cee7abc27092 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:16.323434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dce5a-bf19-428b-b328-cee7abc27092 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:16.323704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dce5a-bf19-428b-b328-cee7abc27092 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:16.332723 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7dce5a-bf19-428b-b328-cee7abc27092 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:16.334383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7dce5a-bf19-428b-b328-cee7abc27092 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:16.346905 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 706/1425] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:17.211151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013092d-feeb-4ada-8bed-8d8ab5cae3c4 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:17.211841 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013092d-feeb-4ada-8bed-8d8ab5cae3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:17.211841 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013092d-feeb-4ada-8bed-8d8ab5cae3c4 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:17.213137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 10:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:17.219551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229e45f-e9f7-4a9d-9cb4-7f668d8cb1ef None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:17.220276 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229e45f-e9f7-4a9d-9cb4-7f668d8cb1ef None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:17.220478 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229e45f-e9f7-4a9d-9cb4-7f668d8cb1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:17.242668 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a229e45f-e9f7-4a9d-9cb4-7f668d8cb1ef None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:17.243874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a229e45f-e9f7-4a9d-9cb4-7f668d8cb1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:17.253255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 10:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:17.409294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013092d-feeb-4ada-8bed-8d8ab5cae3c4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:17.428349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013092d-feeb-4ada-8bed-8d8ab5cae3c4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:17.437524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2bac6-49ad-4268-bb4e-8b22d3617cda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:17.438481 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2bac6-49ad-4268-bb4e-8b22d3617cda service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:17.438829 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2bac6-49ad-4268-bb4e-8b22d3617cda service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:17.439174 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2bac6-49ad-4268-bb4e-8b22d3617cda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:17.450105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2bac6-49ad-4268-bb4e-8b22d3617cda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:17.451711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2bac6-49ad-4268-bb4e-8b22d3617cda service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:17.462089 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 721/1428] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:18.283141 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995ce15-3543-4f39-937f-04677e23e55b None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:18.283710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995ce15-3543-4f39-937f-04677e23e55b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:18.283710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995ce15-3543-4f39-937f-04677e23e55b None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:18.285118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 10:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:18.291058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d0cb4-69de-4553-b627-b2bb1077a8ed None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:18.291992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d0cb4-69de-4553-b627-b2bb1077a8ed None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:18.291992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d0cb4-69de-4553-b627-b2bb1077a8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:18.316777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-031d0cb4-69de-4553-b627-b2bb1077a8ed None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:18.318006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-031d0cb4-69de-4553-b627-b2bb1077a8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:18.327418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 10:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:18.496849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995ce15-3543-4f39-937f-04677e23e55b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:18.513373 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995ce15-3543-4f39-937f-04677e23e55b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:18.519860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e9f1b-eb0e-4697-85ca-26af98a79257 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:18.520498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e9f1b-eb0e-4697-85ca-26af98a79257 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:18.520781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e9f1b-eb0e-4697-85ca-26af98a79257 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:18.521058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e9f1b-eb0e-4697-85ca-26af98a79257 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:18.529656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e9f1b-eb0e-4697-85ca-26af98a79257 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:18.531282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e9f1b-eb0e-4697-85ca-26af98a79257 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:18.540814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 709/1431] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:19.348770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49ab92-7f31-4b45-bd06-711686f6443a None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:19.349373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49ab92-7f31-4b45-bd06-711686f6443a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:19.349373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49ab92-7f31-4b45-bd06-711686f6443a None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:19.350743 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 10:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:19.355892 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e689d6a-17d1-4198-be17-75d5cf39fa66 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:19.356366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e689d6a-17d1-4198-be17-75d5cf39fa66 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:19.356496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e689d6a-17d1-4198-be17-75d5cf39fa66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:19.379120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4e689d6a-17d1-4198-be17-75d5cf39fa66 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:19.380887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4e689d6a-17d1-4198-be17-75d5cf39fa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:19.391082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 10:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:19.543603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49ab92-7f31-4b45-bd06-711686f6443a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:19.560534 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49ab92-7f31-4b45-bd06-711686f6443a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:19.567739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb7e93-ee04-44d7-a302-d35e83c761cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:19.568379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb7e93-ee04-44d7-a302-d35e83c761cc service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:19.568665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb7e93-ee04-44d7-a302-d35e83c761cc service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:19.568943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb7e93-ee04-44d7-a302-d35e83c761cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:19.577723 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb7e93-ee04-44d7-a302-d35e83c761cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:19.579327 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb7e93-ee04-44d7-a302-d35e83c761cc service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:19.589108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 724/1434] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:20.390162 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123db370-b924-4882-b026-9a3ada844427 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:20.390718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123db370-b924-4882-b026-9a3ada844427 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:20.390718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123db370-b924-4882-b026-9a3ada844427 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:20.392666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 10:02:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:20.397763 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ccb65-700d-480c-a2b8-1670dae89f13 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:20.398237 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ccb65-700d-480c-a2b8-1670dae89f13 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:20.398334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ccb65-700d-480c-a2b8-1670dae89f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:20.418399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5e2ccb65-700d-480c-a2b8-1670dae89f13 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:20.419596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5e2ccb65-700d-480c-a2b8-1670dae89f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:20.427965 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 10:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:20.580775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123db370-b924-4882-b026-9a3ada844427 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:20.600747 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123db370-b924-4882-b026-9a3ada844427 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:20.607124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c9c95-245e-4059-bd45-4f60e6ee19b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:20.607785 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c9c95-245e-4059-bd45-4f60e6ee19b4 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:20.608069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c9c95-245e-4059-bd45-4f60e6ee19b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:20.608343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c9c95-245e-4059-bd45-4f60e6ee19b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:20.620513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c9c95-245e-4059-bd45-4f60e6ee19b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:20.622171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c9c95-245e-4059-bd45-4f60e6ee19b4 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:20.631899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 712/1437] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:21.455589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b9224-dc4d-4b00-8d1a-5b4d91956057 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:21.456464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b9224-dc4d-4b00-8d1a-5b4d91956057 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:21.456464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b9224-dc4d-4b00-8d1a-5b4d91956057 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:21.458689 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 10:02:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:21.464364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d303cda-d751-4258-bafe-08977f50bc63 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:21.464815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d303cda-d751-4258-bafe-08977f50bc63 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:21.464930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d303cda-d751-4258-bafe-08977f50bc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:21.485042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4d303cda-d751-4258-bafe-08977f50bc63 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:21.486287 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4d303cda-d751-4258-bafe-08977f50bc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:21.500090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 10:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:21.666797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b9224-dc4d-4b00-8d1a-5b4d91956057 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:21.694035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b9224-dc4d-4b00-8d1a-5b4d91956057 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:21.702484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d2328-f8ce-4dad-819f-683748bb2a30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:21.703120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d2328-f8ce-4dad-819f-683748bb2a30 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:21.703404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d2328-f8ce-4dad-819f-683748bb2a30 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:21.703676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d2328-f8ce-4dad-819f-683748bb2a30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:21.712386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d2328-f8ce-4dad-819f-683748bb2a30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:21.713945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d2328-f8ce-4dad-819f-683748bb2a30 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:21.723817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 727/1440] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:23.632791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77df83-e3ab-43ea-8bd5-9d2c0e757c60 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:23.633403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77df83-e3ab-43ea-8bd5-9d2c0e757c60 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:23.633403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77df83-e3ab-43ea-8bd5-9d2c0e757c60 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:23.634762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:23.643985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4cd16-25fa-48ca-9d77-9312ac4924d5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:23.644687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4cd16-25fa-48ca-9d77-9312ac4924d5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:23.644828 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4cd16-25fa-48ca-9d77-9312ac4924d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:23.666034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-40c4cd16-25fa-48ca-9d77-9312ac4924d5 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:23.667469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-40c4cd16-25fa-48ca-9d77-9312ac4924d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:23.675901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:23.717980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77df83-e3ab-43ea-8bd5-9d2c0e757c60 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:23.731167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77df83-e3ab-43ea-8bd5-9d2c0e757c60 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:23.735343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead7e45-2958-48fc-866d-531811b22a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:23.735990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead7e45-2958-48fc-866d-531811b22a0c service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:23.736269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead7e45-2958-48fc-866d-531811b22a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:23.736553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead7e45-2958-48fc-866d-531811b22a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:23.747762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead7e45-2958-48fc-866d-531811b22a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:23.749253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead7e45-2958-48fc-866d-531811b22a0c service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:23.760334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 715/1443] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:26.043502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ffbb1-65cd-4b23-be50-a9a586b09fcc None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:26.044425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ffbb1-65cd-4b23-be50-a9a586b09fcc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:26.044425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ffbb1-65cd-4b23-be50-a9a586b09fcc None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:26.046514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:02:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:26.053032 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3952312-dc0e-488a-9374-530c1a2d8277 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:26.053808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3952312-dc0e-488a-9374-530c1a2d8277 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:26.053965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3952312-dc0e-488a-9374-530c1a2d8277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:26.074447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b3952312-dc0e-488a-9374-530c1a2d8277 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:26.075662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b3952312-dc0e-488a-9374-530c1a2d8277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:26.084408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:26.125447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ffbb1-65cd-4b23-be50-a9a586b09fcc None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:26.139774 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ffbb1-65cd-4b23-be50-a9a586b09fcc None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:26.143359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9754e48-452e-4b7f-ac25-30f8c964ad84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:26.144042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9754e48-452e-4b7f-ac25-30f8c964ad84 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:26.144326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9754e48-452e-4b7f-ac25-30f8c964ad84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:26.144604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9754e48-452e-4b7f-ac25-30f8c964ad84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:26.154942 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9754e48-452e-4b7f-ac25-30f8c964ad84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:26.156391 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9754e48-452e-4b7f-ac25-30f8c964ad84 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:26.166651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 730/1446] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:28.601327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17108625-209d-43a8-beb3-3fca9005fe65 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:28.601950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17108625-209d-43a8-beb3-3fca9005fe65 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:28.601950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17108625-209d-43a8-beb3-3fca9005fe65 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:28.603291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:28.610160 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7902c-6325-48a3-bcd0-2f563287f9b8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:28.610591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7902c-6325-48a3-bcd0-2f563287f9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:28.610715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7902c-6325-48a3-bcd0-2f563287f9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:28.630751 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-96d7902c-6325-48a3-bcd0-2f563287f9b8 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:28.631952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-96d7902c-6325-48a3-bcd0-2f563287f9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:28.641560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:28.680074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17108625-209d-43a8-beb3-3fca9005fe65 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:28.690839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17108625-209d-43a8-beb3-3fca9005fe65 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:28.694219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cf1a1-dc4e-4e42-825e-1f5167f30642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:28.694849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cf1a1-dc4e-4e42-825e-1f5167f30642 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:28.695126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cf1a1-dc4e-4e42-825e-1f5167f30642 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:28.695398 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cf1a1-dc4e-4e42-825e-1f5167f30642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:28.705164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cf1a1-dc4e-4e42-825e-1f5167f30642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:28.706638 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cf1a1-dc4e-4e42-825e-1f5167f30642 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:28.716022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 718/1449] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:02:30.883562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1bbcb-3a44-4d74-a27f-3eef94ac10a9 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:30.884415 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1bbcb-3a44-4d74-a27f-3eef94ac10a9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:30.884415 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1bbcb-3a44-4d74-a27f-3eef94ac10a9 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:30.886316 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:02:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:30.891641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d910a-9506-431a-9f54-b5f66c60d472 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:30.892084 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d910a-9506-431a-9f54-b5f66c60d472 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:30.892198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d910a-9506-431a-9f54-b5f66c60d472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:30.911097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-715d910a-9506-431a-9f54-b5f66c60d472 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:30.912345 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-715d910a-9506-431a-9f54-b5f66c60d472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:30.921534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:30.960625 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1bbcb-3a44-4d74-a27f-3eef94ac10a9 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:30.974249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1bbcb-3a44-4d74-a27f-3eef94ac10a9 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:30.978279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c646a-747f-4e1c-9ba8-4ca5e738e932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:30.978967 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c646a-747f-4e1c-9ba8-4ca5e738e932 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:30.979247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c646a-747f-4e1c-9ba8-4ca5e738e932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:30.979524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c646a-747f-4e1c-9ba8-4ca5e738e932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:30.988706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c646a-747f-4e1c-9ba8-4ca5e738e932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:30.990182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c646a-747f-4e1c-9ba8-4ca5e738e932 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:30.999628 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 733/1452] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:33.077651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc1d78-91ed-4260-b7e9-040ec62d9be4 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:33.078262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc1d78-91ed-4260-b7e9-040ec62d9be4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:33.078262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc1d78-91ed-4260-b7e9-040ec62d9be4 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:33.079651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:33.086139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f9084-ebc6-4119-9ff3-807e870e35cd None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:33.086844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f9084-ebc6-4119-9ff3-807e870e35cd None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:33.087036 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f9084-ebc6-4119-9ff3-807e870e35cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:33.112113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4e7f9084-ebc6-4119-9ff3-807e870e35cd None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:33.113305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4e7f9084-ebc6-4119-9ff3-807e870e35cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:33.122745 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:33.163997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc1d78-91ed-4260-b7e9-040ec62d9be4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:33.177291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc1d78-91ed-4260-b7e9-040ec62d9be4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:33.181587 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee40be-e7f9-4a70-b404-e14941865ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:33.182279 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee40be-e7f9-4a70-b404-e14941865ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:33.182585 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee40be-e7f9-4a70-b404-e14941865ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:33.182842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee40be-e7f9-4a70-b404-e14941865ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:33.192911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee40be-e7f9-4a70-b404-e14941865ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:33.194391 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee40be-e7f9-4a70-b404-e14941865ec2 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:33.204152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 721/1455] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:35.350879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8728b2-bed5-4e03-959c-2f32a79cc483 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:35.351740 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8728b2-bed5-4e03-959c-2f32a79cc483 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:35.351740 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8728b2-bed5-4e03-959c-2f32a79cc483 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:35.353915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:02:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:35.361751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0f3a4-0390-4b64-9afe-2f016279905d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:35.362200 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0f3a4-0390-4b64-9afe-2f016279905d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:35.362296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0f3a4-0390-4b64-9afe-2f016279905d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:35.383994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-99d0f3a4-0390-4b64-9afe-2f016279905d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:35.385911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-99d0f3a4-0390-4b64-9afe-2f016279905d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:35.395584 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:35.433642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8728b2-bed5-4e03-959c-2f32a79cc483 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:35.450944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8728b2-bed5-4e03-959c-2f32a79cc483 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:35.455586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8106074-e54e-4e69-84d7-c20ec23e8b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:35.456235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8106074-e54e-4e69-84d7-c20ec23e8b01 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:35.456521 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8106074-e54e-4e69-84d7-c20ec23e8b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:35.456804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8106074-e54e-4e69-84d7-c20ec23e8b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:35.466443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8106074-e54e-4e69-84d7-c20ec23e8b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:35.467892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8106074-e54e-4e69-84d7-c20ec23e8b01 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:35.477581 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 736/1458] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:37.553537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3ddb7-c3f4-4f86-902d-e3a5ae382097 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:37.554127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3ddb7-c3f4-4f86-902d-e3a5ae382097 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:37.554127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3ddb7-c3f4-4f86-902d-e3a5ae382097 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:37.555535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:37.561303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9159c-da1d-4735-bdda-3c5dca7cd5fb None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:37.561778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9159c-da1d-4735-bdda-3c5dca7cd5fb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:37.561908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9159c-da1d-4735-bdda-3c5dca7cd5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:37.581918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-62b9159c-da1d-4735-bdda-3c5dca7cd5fb None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:37.583130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-62b9159c-da1d-4735-bdda-3c5dca7cd5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:37.592259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:02:37.629766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3ddb7-c3f4-4f86-902d-e3a5ae382097 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:37.645676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3ddb7-c3f4-4f86-902d-e3a5ae382097 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:37.654992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206aaa24-5976-46f7-8367-175abd8d920c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:37.655644 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aaa24-5976-46f7-8367-175abd8d920c service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:37.655908 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aaa24-5976-46f7-8367-175abd8d920c service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:37.656184 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aaa24-5976-46f7-8367-175abd8d920c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:37.666691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206aaa24-5976-46f7-8367-175abd8d920c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:37.668232 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206aaa24-5976-46f7-8367-175abd8d920c service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:37.677756 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 724/1461] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:39.742774 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee6215-c2b0-4453-a186-a355c2614b81 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:39.743375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee6215-c2b0-4453-a186-a355c2614b81 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:39.743375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee6215-c2b0-4453-a186-a355c2614b81 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:39.745407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:02:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:39.754589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c1e60-acd9-4b2f-9f1b-99f894c37f8e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:39.755010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c1e60-acd9-4b2f-9f1b-99f894c37f8e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:39.755179 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c1e60-acd9-4b2f-9f1b-99f894c37f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:39.763965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=79 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:02:39.779440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-977c1e60-acd9-4b2f-9f1b-99f894c37f8e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:39.780657 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-977c1e60-acd9-4b2f-9f1b-99f894c37f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:39.793389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:39.833255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee6215-c2b0-4453-a186-a355c2614b81 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:39.839550 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=73 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:02:39.847632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee6215-c2b0-4453-a186-a355c2614b81 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:39.852105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c9a45-d56b-42ba-a21d-27e0633e12d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:39.852747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c9a45-d56b-42ba-a21d-27e0633e12d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:39.853031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c9a45-d56b-42ba-a21d-27e0633e12d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:39.853308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c9a45-d56b-42ba-a21d-27e0633e12d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:39.863996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c9a45-d56b-42ba-a21d-27e0633e12d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:39.865994 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c9a45-d56b-42ba-a21d-27e0633e12d1 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:39.877394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 739/1464] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:41.973905 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec434669-45d7-4378-9ae8-0e575d62ab09 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:41.974524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec434669-45d7-4378-9ae8-0e575d62ab09 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:41.974524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec434669-45d7-4378-9ae8-0e575d62ab09 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:41.975853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:41.982231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82b70f-b472-4d98-839f-edd96887ba4a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:41.982982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82b70f-b472-4d98-839f-edd96887ba4a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:41.983125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82b70f-b472-4d98-839f-edd96887ba4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:42.006670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-dd82b70f-b472-4d98-839f-edd96887ba4a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:42.007866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-dd82b70f-b472-4d98-839f-edd96887ba4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:42.017165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:42.058239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec434669-45d7-4378-9ae8-0e575d62ab09 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:42.071637 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec434669-45d7-4378-9ae8-0e575d62ab09 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:42.076638 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7f822-ffd9-4e9d-bf9d-e930c6457e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:42.077294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7f822-ffd9-4e9d-bf9d-e930c6457e6f service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:42.077552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7f822-ffd9-4e9d-bf9d-e930c6457e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:42.077885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7f822-ffd9-4e9d-bf9d-e930c6457e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:42.087848 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7f822-ffd9-4e9d-bf9d-e930c6457e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:42.089876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7f822-ffd9-4e9d-bf9d-e930c6457e6f service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:42.099811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 727/1467] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:02:44.169264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba072a-5b88-4bde-9635-a90c620aab11 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:44.170129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba072a-5b88-4bde-9635-a90c620aab11 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:44.170325 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba072a-5b88-4bde-9635-a90c620aab11 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:44.172465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:02:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:02:44.179001 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2973356-debc-4182-b6cf-934d6d099e6a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:44.179472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2973356-debc-4182-b6cf-934d6d099e6a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:44.179568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2973356-debc-4182-b6cf-934d6d099e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:44.204921 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a2973356-debc-4182-b6cf-934d6d099e6a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:02:44.206167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a2973356-debc-4182-b6cf-934d6d099e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:02:44.216031 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:02:44.256149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba072a-5b88-4bde-9635-a90c620aab11 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:02:44.269203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba072a-5b88-4bde-9635-a90c620aab11 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:02:44.274051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500dd31-6124-47d4-9ba4-586a18874066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:02:44.274696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500dd31-6124-47d4-9ba4-586a18874066 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:02:44.274980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500dd31-6124-47d4-9ba4-586a18874066 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:02:44.275258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500dd31-6124-47d4-9ba4-586a18874066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:02:44.285784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500dd31-6124-47d4-9ba4-586a18874066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:02:44.287245 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500dd31-6124-47d4-9ba4-586a18874066 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:02:44.296860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 742/1470] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:02:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:02:54.199544 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=11 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:02:54.292535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=14 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:03:04.114044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f2fde-ca8f-40fc-8664-240498a26d8c None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:04.114649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f2fde-ca8f-40fc-8664-240498a26d8c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:04.114649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f2fde-ca8f-40fc-8664-240498a26d8c None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:04.116001 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 729/1471] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:03:04.122140 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331299b7-9736-4f80-af4f-ea46461082e4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:04.122585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331299b7-9736-4f80-af4f-ea46461082e4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:04.122694 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331299b7-9736-4f80-af4f-ea46461082e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:04.143171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-331299b7-9736-4f80-af4f-ea46461082e4 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:03:04.144390 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-331299b7-9736-4f80-af4f-ea46461082e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:03:04.155095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 743/1472] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:03:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:03:04.251208 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f9036-5a82-4d4f-a890-20fa43ee5d56 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:04.251523 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f9036-5a82-4d4f-a890-20fa43ee5d56 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:04.251803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f9036-5a82-4d4f-a890-20fa43ee5d56 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:04.253222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 730/1473] 10.222.0.79 () {54 vars in 962 bytes} [Mon Jul 4 10:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:03:04.258350 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7559ef-ced7-441d-b28e-5ca31df03884 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:04.258634 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7559ef-ced7-441d-b28e-5ca31df03884 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:04.258911 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7559ef-ced7-441d-b28e-5ca31df03884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:04.278596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2b7559ef-ced7-441d-b28e-5ca31df03884 None None] MFA Rules not processed for user `fef685a17c804e2bbfea1e07cfdf5cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:03:04.279780 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2b7559ef-ced7-441d-b28e-5ca31df03884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:03:04.287238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 744/1474] 10.222.0.79 () {58 vars in 1059 bytes} [Mon Jul 4 10:03:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:03:04.294664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f9036-5a82-4d4f-a890-20fa43ee5d56 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:03:04.306426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f9036-5a82-4d4f-a890-20fa43ee5d56 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:03:04.310545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe2907-7518-4924-b3e6-0e5e9f30747b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef685a17c804e2bbfea1e07cfdf5cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:03:04.311166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe2907-7518-4924-b3e6-0e5e9f30747b service glance] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:04.311444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe2907-7518-4924-b3e6-0e5e9f30747b service glance] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:04.311715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe2907-7518-4924-b3e6-0e5e9f30747b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:04.320787 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe2907-7518-4924-b3e6-0e5e9f30747b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:03:04.322273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe2907-7518-4924-b3e6-0e5e9f30747b service glance] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:03:04.331888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 731/1475] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:03:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:03:06.495333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7637a-06b3-4913-a13d-0938fb271a57 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:06.495879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7637a-06b3-4913-a13d-0938fb271a57 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:06.496158 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7637a-06b3-4913-a13d-0938fb271a57 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:06.497565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 745/1476] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:03:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:03:06.503958 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395512e-fd24-425f-9cd4-057a92c21d9e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:06.504771 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395512e-fd24-425f-9cd4-057a92c21d9e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:06.504989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395512e-fd24-425f-9cd4-057a92c21d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:06.532470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-0395512e-fd24-425f-9cd4-057a92c21d9e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:03:06.533673 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0395512e-fd24-425f-9cd4-057a92c21d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:03:06.542843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 732/1477] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:03:06.567515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7637a-06b3-4913-a13d-0938fb271a57 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:03:06.581711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7637a-06b3-4913-a13d-0938fb271a57 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:03:06.586525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8701f1-166b-44c3-86c0-d47fc331e129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:03:06.587179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8701f1-166b-44c3-86c0-d47fc331e129 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:06.587465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8701f1-166b-44c3-86c0-d47fc331e129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:06.587742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8701f1-166b-44c3-86c0-d47fc331e129 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:06.588484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8701f1-166b-44c3-86c0-d47fc331e129 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:03:06.590575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-eb8701f1-166b-44c3-86c0-d47fc331e129 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:03:06.590575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jul 04 10:03:06.590575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: Jul 04 10:03:06.591741 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8701f1-166b-44c3-86c0-d47fc331e129 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:03:06.612548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 746/1478] 10.222.0.79 () {56 vars in 1108 bytes} [Mon Jul 4 10:03:06 2022] GET /identity/v3/projects => generated 2553 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:03:08.661478 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22be19-8a77-41ba-b4ee-73771d303d29 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:08.662444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22be19-8a77-41ba-b4ee-73771d303d29 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:08.662444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22be19-8a77-41ba-b4ee-73771d303d29 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:08.664696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 733/1479] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:03:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:03:08.671240 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df38bc5-75b5-470f-97b0-f2902d1e2d4e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:08.671973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df38bc5-75b5-470f-97b0-f2902d1e2d4e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:08.672122 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df38bc5-75b5-470f-97b0-f2902d1e2d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:08.694069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3df38bc5-75b5-470f-97b0-f2902d1e2d4e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:03:08.695210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3df38bc5-75b5-470f-97b0-f2902d1e2d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:03:08.705330 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 747/1480] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:03:08.750212 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22be19-8a77-41ba-b4ee-73771d303d29 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:03:08.766209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22be19-8a77-41ba-b4ee-73771d303d29 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:03:08.772015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667b804-c42c-41cc-b7b8-4eeb553a213a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:03:08.772696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667b804-c42c-41cc-b7b8-4eeb553a213a service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:08.772963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667b804-c42c-41cc-b7b8-4eeb553a213a service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:08.773240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667b804-c42c-41cc-b7b8-4eeb553a213a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:08.781804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667b804-c42c-41cc-b7b8-4eeb553a213a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:03:08.783403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667b804-c42c-41cc-b7b8-4eeb553a213a service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:03:08.792702 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 734/1481] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:03:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:03:11.091177 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c077a-11b9-49c5-babb-2d79daee1b75 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:11.091663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c077a-11b9-49c5-babb-2d79daee1b75 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:11.091775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c077a-11b9-49c5-babb-2d79daee1b75 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:11.093122 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 748/1482] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:03:11.099779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97a81a-9eb5-43b7-aff4-b7cc889989c3 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:11.100237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97a81a-9eb5-43b7-aff4-b7cc889989c3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:11.100341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97a81a-9eb5-43b7-aff4-b7cc889989c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:11.122556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6d97a81a-9eb5-43b7-aff4-b7cc889989c3 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:03:11.123767 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6d97a81a-9eb5-43b7-aff4-b7cc889989c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:03:11.135354 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 735/1483] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:03:11.173040 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c077a-11b9-49c5-babb-2d79daee1b75 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:03:11.186081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c077a-11b9-49c5-babb-2d79daee1b75 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:03:11.192920 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e704267-d5b3-4581-8153-3f8f113f9845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:03:11.193935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704267-d5b3-4581-8153-3f8f113f9845 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:11.194383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704267-d5b3-4581-8153-3f8f113f9845 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:11.194812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704267-d5b3-4581-8153-3f8f113f9845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:11.206543 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e704267-d5b3-4581-8153-3f8f113f9845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:03:11.207985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e704267-d5b3-4581-8153-3f8f113f9845 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:03:11.217907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 749/1484] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:03:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:03:13.310832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbef3b-0b93-4d4d-9ca5-ec3d0f8a01d7 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:13.311457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbef3b-0b93-4d4d-9ca5-ec3d0f8a01d7 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:13.311457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbef3b-0b93-4d4d-9ca5-ec3d0f8a01d7 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:13.312788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 736/1485] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 10:03:13.318228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140ed7e-b5b1-40a9-af38-0a6665a92ac7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:13.318678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140ed7e-b5b1-40a9-af38-0a6665a92ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:13.318779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140ed7e-b5b1-40a9-af38-0a6665a92ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:13.337539 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c140ed7e-b5b1-40a9-af38-0a6665a92ac7 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:03:13.338770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c140ed7e-b5b1-40a9-af38-0a6665a92ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:03:13.348958 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 750/1486] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:03:13.387770 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbef3b-0b93-4d4d-9ca5-ec3d0f8a01d7 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:03:13.400288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbef3b-0b93-4d4d-9ca5-ec3d0f8a01d7 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:03:13.404979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa6dcd-4076-401c-8975-b1e43cca685f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:03:13.405703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6dcd-4076-401c-8975-b1e43cca685f service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:13.405921 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6dcd-4076-401c-8975-b1e43cca685f service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:13.406211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6dcd-4076-401c-8975-b1e43cca685f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:13.415687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa6dcd-4076-401c-8975-b1e43cca685f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:03:13.417133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa6dcd-4076-401c-8975-b1e43cca685f service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:03:13.426575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 737/1487] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:03:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:03:15.588975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0db24-de1a-408c-99a2-06a6fce3a3a3 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:15.589588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0db24-de1a-408c-99a2-06a6fce3a3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:15.589588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0db24-de1a-408c-99a2-06a6fce3a3a3 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:15.590978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 751/1488] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:03:15.598470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f6471-25fb-42b8-bc45-00fb694b75cb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:15.598987 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f6471-25fb-42b8-bc45-00fb694b75cb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:15.599106 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f6471-25fb-42b8-bc45-00fb694b75cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:15.627731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-091f6471-25fb-42b8-bc45-00fb694b75cb None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:03:15.629516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-091f6471-25fb-42b8-bc45-00fb694b75cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:03:15.641546 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 738/1489] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:03:15.683014 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0db24-de1a-408c-99a2-06a6fce3a3a3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:03:15.701201 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0db24-de1a-408c-99a2-06a6fce3a3a3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:03:15.707274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa74f5-c6f7-4b38-8ad3-9542cd5f6c1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:03:15.708372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa74f5-c6f7-4b38-8ad3-9542cd5f6c1c service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:15.708891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa74f5-c6f7-4b38-8ad3-9542cd5f6c1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:15.709402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa74f5-c6f7-4b38-8ad3-9542cd5f6c1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:15.726691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa74f5-c6f7-4b38-8ad3-9542cd5f6c1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:03:15.729463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa74f5-c6f7-4b38-8ad3-9542cd5f6c1c service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:03:15.741264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 752/1490] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:03:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:03:17.952803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8129ac-1c58-4615-9574-5c32be7cf699 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:17.953809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8129ac-1c58-4615-9574-5c32be7cf699 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:17.953809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8129ac-1c58-4615-9574-5c32be7cf699 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:17.955866 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 739/1491] 10.222.0.79 () {54 vars in 926 bytes} [Mon Jul 4 10:03:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:03:17.964879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1034f-b0a5-4778-934c-7cf8bd75a34d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:17.966017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1034f-b0a5-4778-934c-7cf8bd75a34d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:17.966441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1034f-b0a5-4778-934c-7cf8bd75a34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:17.991039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-70b1034f-b0a5-4778-934c-7cf8bd75a34d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:03:17.992254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-70b1034f-b0a5-4778-934c-7cf8bd75a34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:03:18.002035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 753/1492] 10.222.0.79 () {58 vars in 1023 bytes} [Mon Jul 4 10:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:03:18.042148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8129ac-1c58-4615-9574-5c32be7cf699 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:03:18.055243 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8129ac-1c58-4615-9574-5c32be7cf699 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:03:18.059602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb6b1c-7de9-4c2b-b8a7-cf9a1419d41f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:03:18.060237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb6b1c-7de9-4c2b-b8a7-cf9a1419d41f service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:03:18.060515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb6b1c-7de9-4c2b-b8a7-cf9a1419d41f service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:03:18.060786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb6b1c-7de9-4c2b-b8a7-cf9a1419d41f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:03:18.070823 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb6b1c-7de9-4c2b-b8a7-cf9a1419d41f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:03:18.072257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb6b1c-7de9-4c2b-b8a7-cf9a1419d41f service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:03:18.081811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 740/1493] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:03:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:03:27.985767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=33 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:03:28.077513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=33 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:04:46.799884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39573fc7-6c7b-476d-b9ac-a5bde9c0e461 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:46.800799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39573fc7-6c7b-476d-b9ac-a5bde9c0e461 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:46.800799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39573fc7-6c7b-476d-b9ac-a5bde9c0e461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:46.825040 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-39573fc7-6c7b-476d-b9ac-a5bde9c0e461 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:04:46.826951 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-39573fc7-6c7b-476d-b9ac-a5bde9c0e461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:04:46.841175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 754/1494] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:04:46.849826 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a7eb0-0811-4165-b4df-100c9bffc1c9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:46.850349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a7eb0-0811-4165-b4df-100c9bffc1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:46.850349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a7eb0-0811-4165-b4df-100c9bffc1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:46.870229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4e9a7eb0-0811-4165-b4df-100c9bffc1c9 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:04:46.871980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4e9a7eb0-0811-4165-b4df-100c9bffc1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:04:46.884468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 741/1495] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:04:46.890914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39573fc7-6c7b-476d-b9ac-a5bde9c0e461 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:04:46.901873 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39573fc7-6c7b-476d-b9ac-a5bde9c0e461 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:04:46.907229 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f7112-edc8-4759-9d28-43f4293f066b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:04:46.907885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f7112-edc8-4759-9d28-43f4293f066b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:46.908164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f7112-edc8-4759-9d28-43f4293f066b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:46.908440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f7112-edc8-4759-9d28-43f4293f066b admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:46.909166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f7112-edc8-4759-9d28-43f4293f066b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:04:46.909470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f7112-edc8-4759-9d28-43f4293f066b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:04:46.910515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-213f7112-edc8-4759-9d28-43f4293f066b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:04:46.910515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jul 04 10:04:46.910515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: Jul 04 10:04:46.911375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f7112-edc8-4759-9d28-43f4293f066b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:04:46.923078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 755/1496] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:04:46.931256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a7eb0-0811-4165-b4df-100c9bffc1c9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:04:46.941242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a7eb0-0811-4165-b4df-100c9bffc1c9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:04:46.947176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f4ec4-554c-4f04-bd94-090a4d1a0c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:04:46.947850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f4ec4-554c-4f04-bd94-090a4d1a0c21 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:46.948131 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f4ec4-554c-4f04-bd94-090a4d1a0c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:46.948405 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f4ec4-554c-4f04-bd94-090a4d1a0c21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:46.949081 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f4ec4-554c-4f04-bd94-090a4d1a0c21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:04:46.951242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-592f4ec4-554c-4f04-bd94-090a4d1a0c21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:04:46.951242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jul 04 10:04:46.951242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: Jul 04 10:04:46.952446 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f4ec4-554c-4f04-bd94-090a4d1a0c21 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:04:46.990173 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 742/1497] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:04:46 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:04:46.994567 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f7112-edc8-4759-9d28-43f4293f066b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:04:47.004868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f7112-edc8-4759-9d28-43f4293f066b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:04:47.064931 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f6eff-842a-45bb-a372-93dfb6dba202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:04:47.065572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f6eff-842a-45bb-a372-93dfb6dba202 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:47.065895 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f6eff-842a-45bb-a372-93dfb6dba202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:47.066171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f6eff-842a-45bb-a372-93dfb6dba202 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:47.066865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f6eff-842a-45bb-a372-93dfb6dba202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:04:47.067937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-889f6eff-842a-45bb-a372-93dfb6dba202 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:04:47.067937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jul 04 10:04:47.067937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: Jul 04 10:04:47.069203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f6eff-842a-45bb-a372-93dfb6dba202 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:04:47.091349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 756/1498] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:04:46 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:04:47.095636 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f4ec4-554c-4f04-bd94-090a4d1a0c21 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:04:47.107013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f4ec4-554c-4f04-bd94-090a4d1a0c21 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:04:47.112314 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde49a4-38a3-42e8-9c38-e713ccfd2bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:04:47.112976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde49a4-38a3-42e8-9c38-e713ccfd2bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:47.113233 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde49a4-38a3-42e8-9c38-e713ccfd2bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:47.113503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde49a4-38a3-42e8-9c38-e713ccfd2bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:47.114265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde49a4-38a3-42e8-9c38-e713ccfd2bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:04:47.115303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-7cde49a4-38a3-42e8-9c38-e713ccfd2bd3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:04:47.115303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jul 04 10:04:47.115303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: Jul 04 10:04:47.116195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde49a4-38a3-42e8-9c38-e713ccfd2bd3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:04:47.133120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 743/1499] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:04:47.134493 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f6eff-842a-45bb-a372-93dfb6dba202 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:04:47.145455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f6eff-842a-45bb-a372-93dfb6dba202 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:04:47.151119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98b350-5729-4d5d-b91d-e99ff9ebc171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:04:47.151789 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98b350-5729-4d5d-b91d-e99ff9ebc171 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:47.152074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98b350-5729-4d5d-b91d-e99ff9ebc171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:47.152346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98b350-5729-4d5d-b91d-e99ff9ebc171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:47.153082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98b350-5729-4d5d-b91d-e99ff9ebc171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:04:47.154441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-db98b350-5729-4d5d-b91d-e99ff9ebc171 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:04:47.154441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jul 04 10:04:47.154441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: Jul 04 10:04:47.155350 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98b350-5729-4d5d-b91d-e99ff9ebc171 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:04:47.169212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 757/1500] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:04:47.175372 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde49a4-38a3-42e8-9c38-e713ccfd2bd3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:04:47.191344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde49a4-38a3-42e8-9c38-e713ccfd2bd3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:04:47.200105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c8db8-c86d-497d-9819-4843f176b051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:04:47.201068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c8db8-c86d-497d-9819-4843f176b051 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:47.201347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c8db8-c86d-497d-9819-4843f176b051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:47.201648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c8db8-c86d-497d-9819-4843f176b051 admin admin] PATH_INFO: `/v3/projects/88d1c0a220914c7da48e7d18b798d9b5/users/b27d9777464345a599b3d069ff8571cf/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:47.217185 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c8db8-c86d-497d-9819-4843f176b051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88d1c0a220914c7da48e7d18b798d9b5, user_id=b27d9777464345a599b3d069ff8571cf, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:04:47.218269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-ac7c8db8-c86d-497d-9819-4843f176b051 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:04:47.218269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jul 04 10:04:47.218269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: Jul 04 10:04:47.220209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c8db8-c86d-497d-9819-4843f176b051 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:04:47.236282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 744/1501] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:04:47 2022] PUT /identity/v3/projects/88d1c0a220914c7da48e7d18b798d9b5/users/b27d9777464345a599b3d069ff8571cf/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:04:47.249003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0398b7-bb0e-4d0a-9d0d-3870d750abd8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:47.249299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0398b7-bb0e-4d0a-9d0d-3870d750abd8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:47.249603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0398b7-bb0e-4d0a-9d0d-3870d750abd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:47.277796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-be0398b7-bb0e-4d0a-9d0d-3870d750abd8 None None] MFA Rules not processed for user `b27d9777464345a599b3d069ff8571cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:04:47.279261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-be0398b7-bb0e-4d0a-9d0d-3870d750abd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:04:47.355851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 758/1502] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:04:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:04:47.369468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3448ee-1f7e-49b4-9327-3a2f2db6de29 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:47.369791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3448ee-1f7e-49b4-9327-3a2f2db6de29 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:47.370075 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3448ee-1f7e-49b4-9327-3a2f2db6de29 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:47.371444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 745/1503] 10.222.0.79 () {54 vars in 963 bytes} [Mon Jul 4 10:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:04:47.376498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e8e13-2e26-42f0-add5-e8c57d2c9f82 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:47.376794 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e8e13-2e26-42f0-add5-e8c57d2c9f82 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:47.377076 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e8e13-2e26-42f0-add5-e8c57d2c9f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:47.396109 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a55e8e13-2e26-42f0-add5-e8c57d2c9f82 None None] MFA Rules not processed for user `c5a4af83902d4370a832ed3ea3861b56`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:04:47.397333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a55e8e13-2e26-42f0-add5-e8c57d2c9f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:04:47.444776 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:04:47.454399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 759/1504] 10.222.0.79 () {58 vars in 1060 bytes} [Mon Jul 4 10:04:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:04:47.466080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3448ee-1f7e-49b4-9327-3a2f2db6de29 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:04:47.478092 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3448ee-1f7e-49b4-9327-3a2f2db6de29 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:04:47.481836 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f325b1d-9c6f-4921-967e-663617d198d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:04:47.482494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f325b1d-9c6f-4921-967e-663617d198d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:47.482773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f325b1d-9c6f-4921-967e-663617d198d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:47.483048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f325b1d-9c6f-4921-967e-663617d198d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:47.491360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f325b1d-9c6f-4921-967e-663617d198d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:04:47.492820 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f325b1d-9c6f-4921-967e-663617d198d5 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:04:47.501544 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 746/1505] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:04:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:04:47.548591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d633-8b26-4de0-8d08-13c786e566b4 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:47.548885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d633-8b26-4de0-8d08-13c786e566b4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:47.549167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d633-8b26-4de0-8d08-13c786e566b4 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:47.550515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 760/1506] 10.222.0.79 () {52 vars in 829 bytes} [Mon Jul 4 10:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:04:47.554327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f325b1d-9c6f-4921-967e-663617d198d5 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:04:47.633979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f325b1d-9c6f-4921-967e-663617d198d5 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:04:47.638675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:04:47.639749 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:47.640040 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:47.640324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] PATH_INFO: `/v3/users/b27d9777464345a599b3d069ff8571cf` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:47.641540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b27d9777464345a599b3d069ff8571cf)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:04:47.642621 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:04:47.642621 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jul 04 10:04:47.642621 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: Jul 04 10:04:47.643833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:04:47.715296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b27d9777464345a599b3d069ff8571cf'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:04:47.716053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b27d9777464345a599b3d069ff8571cf'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:04:47.728788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b27d9777464345a599b3d069ff8571cf'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:04:47.750989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b27d9777464345a599b3d069ff8571cf'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:04:47.752153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b27d9777464345a599b3d069ff8571cf'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:04:47.765604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:04:47.778068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 747/1507] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:04:47 2022] DELETE /identity/v3/users/b27d9777464345a599b3d069ff8571cf => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:04:47.783823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b829afa-5612-48c3-9aea-52eb0918cbc3 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:47.784111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b829afa-5612-48c3-9aea-52eb0918cbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:47.784386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b829afa-5612-48c3-9aea-52eb0918cbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:47.802077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9b829afa-5612-48c3-9aea-52eb0918cbc3 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:04:47.803262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9b829afa-5612-48c3-9aea-52eb0918cbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:04:47.864594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 761/1508] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:04:47.874854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:04:47.876418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:04:47.935616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642bce2-059b-4cbb-b25c-81af3d6bf3a3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:04:47.940699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656e4ad-92fb-445b-b75c-aed7d9873a31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:04:47.941376 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656e4ad-92fb-445b-b75c-aed7d9873a31 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:47.941645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656e4ad-92fb-445b-b75c-aed7d9873a31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:47.941938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656e4ad-92fb-445b-b75c-aed7d9873a31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:47.952426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656e4ad-92fb-445b-b75c-aed7d9873a31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:04:47.953917 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656e4ad-92fb-445b-b75c-aed7d9873a31 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:04:47.963699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 748/1509] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:04:48.315969 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b829afa-5612-48c3-9aea-52eb0918cbc3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:04:48.317414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9b829afa-5612-48c3-9aea-52eb0918cbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:04:48.330834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b829afa-5612-48c3-9aea-52eb0918cbc3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:04:48.336217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40476fa7-f20c-4320-8f5d-c6699e52c8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:04:48.337091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40476fa7-f20c-4320-8f5d-c6699e52c8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:04:48.337372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40476fa7-f20c-4320-8f5d-c6699e52c8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:04:48.337674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40476fa7-f20c-4320-8f5d-c6699e52c8b5 admin admin] PATH_INFO: `/v3/projects/88d1c0a220914c7da48e7d18b798d9b5` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:04:48.338902 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40476fa7-f20c-4320-8f5d-c6699e52c8b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88d1c0a220914c7da48e7d18b798d9b5)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:04:48.339935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-40476fa7-f20c-4320-8f5d-c6699e52c8b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:04:48.339935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jul 04 10:04:48.339935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: Jul 04 10:04:48.341157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40476fa7-f20c-4320-8f5d-c6699e52c8b5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:04:48.407230 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-40476fa7-f20c-4320-8f5d-c6699e52c8b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88d1c0a220914c7da48e7d18b798d9b5', 'request_id': 'req-40476fa7-f20c-4320-8f5d-c6699e52c8b5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:04:48.420704 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-40476fa7-f20c-4320-8f5d-c6699e52c8b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88d1c0a220914c7da48e7d18b798d9b5', 'request_id': 'req-40476fa7-f20c-4320-8f5d-c6699e52c8b5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:04:48.427275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-40476fa7-f20c-4320-8f5d-c6699e52c8b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88d1c0a220914c7da48e7d18b798d9b5', 'request_id': 'req-40476fa7-f20c-4320-8f5d-c6699e52c8b5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:04:48.442706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-40476fa7-f20c-4320-8f5d-c6699e52c8b5 admin admin] The token cache is being invalidate because project 88d1c0a220914c7da48e7d18b798d9b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:04:48.442998 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-40476fa7-f20c-4320-8f5d-c6699e52c8b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:04:48.443615 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-40476fa7-f20c-4320-8f5d-c6699e52c8b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:04:48.445310 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 762/1510] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:04:48 2022] DELETE /identity/v3/projects/88d1c0a220914c7da48e7d18b798d9b5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:04:57.959319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=29 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:04:58.439680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:06:26.723350 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba11784-f7a0-4683-9917-9e3a5d004031 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:26.723981 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba11784-f7a0-4683-9917-9e3a5d004031 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:26.723981 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba11784-f7a0-4683-9917-9e3a5d004031 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:26.726087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 10:06:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:06:26.734091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0db19b-4844-481b-96a1-4a3c4f6fa590 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:26.734521 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0db19b-4844-481b-96a1-4a3c4f6fa590 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:26.734813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0db19b-4844-481b-96a1-4a3c4f6fa590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:26.761655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0f0db19b-4844-481b-96a1-4a3c4f6fa590 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:06:26.762875 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0f0db19b-4844-481b-96a1-4a3c4f6fa590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:26.826816 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 10:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:06:26.979931 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba11784-f7a0-4683-9917-9e3a5d004031 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:06:26.982776 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-bba11784-f7a0-4683-9917-9e3a5d004031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:27.074970 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba11784-f7a0-4683-9917-9e3a5d004031 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:06:27.081915 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e5454-eee0-4b4a-b5a1-8d8acadfb51e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:06:27.082507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e5454-eee0-4b4a-b5a1-8d8acadfb51e service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:27.082795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e5454-eee0-4b4a-b5a1-8d8acadfb51e service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:27.083108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e5454-eee0-4b4a-b5a1-8d8acadfb51e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:27.091659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e5454-eee0-4b4a-b5a1-8d8acadfb51e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:06:27.093274 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e5454-eee0-4b4a-b5a1-8d8acadfb51e service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:06:27.103382 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 750/1513] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:06:27.960600 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f650b-5f1f-4d69-bd87-a0d7162d6b87 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:27.961154 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f650b-5f1f-4d69-bd87-a0d7162d6b87 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:27.961154 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f650b-5f1f-4d69-bd87-a0d7162d6b87 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:27.962498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 10:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:06:27.968418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f9f3b-2c7d-401e-bd4e-6552a21f58a8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:27.969166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f9f3b-2c7d-401e-bd4e-6552a21f58a8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:27.969345 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f9f3b-2c7d-401e-bd4e-6552a21f58a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:27.995554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-db4f9f3b-2c7d-401e-bd4e-6552a21f58a8 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:06:27.996764 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-db4f9f3b-2c7d-401e-bd4e-6552a21f58a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:28.006308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 10:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:06:28.159356 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f650b-5f1f-4d69-bd87-a0d7162d6b87 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:06:28.174464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f650b-5f1f-4d69-bd87-a0d7162d6b87 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:06:28.180452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962a2a9-cfc2-477b-892d-4cc9ed9a2fe9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:06:28.181085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962a2a9-cfc2-477b-892d-4cc9ed9a2fe9 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:28.181365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962a2a9-cfc2-477b-892d-4cc9ed9a2fe9 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:28.181635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962a2a9-cfc2-477b-892d-4cc9ed9a2fe9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:28.190291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962a2a9-cfc2-477b-892d-4cc9ed9a2fe9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:06:28.191870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962a2a9-cfc2-477b-892d-4cc9ed9a2fe9 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:06:28.201299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 765/1516] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:06:28.727537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d97e8-c48d-4d42-a412-89831bff39f7 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:28.727806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d97e8-c48d-4d42-a412-89831bff39f7 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:28.728091 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d97e8-c48d-4d42-a412-89831bff39f7 None None] PATH_INFO: `/v3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:28.729475 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 752/1517] 10.222.0.74 () {54 vars in 925 bytes} [Mon Jul 4 10:06:28 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:06:28.737879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ab7a1-e8a7-4790-87af-6cad01c0b2f2 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:28.738174 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ab7a1-e8a7-4790-87af-6cad01c0b2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:28.738464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ab7a1-e8a7-4790-87af-6cad01c0b2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:28.758570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-da9ab7a1-e8a7-4790-87af-6cad01c0b2f2 None None] MFA Rules not processed for user `5dc48318ab7944c197bf5f7673511931`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:06:28.759790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-da9ab7a1-e8a7-4790-87af-6cad01c0b2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:28.830079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 766/1518] 10.222.0.74 () {58 vars in 1015 bytes} [Mon Jul 4 10:06:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:06:32.647090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b0d5f-08b8-4a21-9760-4634aa6f0ea5 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:32.647688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b0d5f-08b8-4a21-9760-4634aa6f0ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:32.647688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b0d5f-08b8-4a21-9760-4634aa6f0ea5 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:32.649050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 753/1519] 10.222.0.79 () {54 vars in 960 bytes} [Mon Jul 4 10:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:06:32.657387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a36e8-69d9-4461-9569-ddf778ad130e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:32.658130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a36e8-69d9-4461-9569-ddf778ad130e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:32.658250 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a36e8-69d9-4461-9569-ddf778ad130e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:32.679470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c92a36e8-69d9-4461-9569-ddf778ad130e None None] MFA Rules not processed for user `5dc48318ab7944c197bf5f7673511931`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:06:32.680661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c92a36e8-69d9-4461-9569-ddf778ad130e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:32.691014 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 767/1520] 10.222.0.79 () {58 vars in 1057 bytes} [Mon Jul 4 10:06:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:06:32.698853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b0d5f-08b8-4a21-9760-4634aa6f0ea5 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:06:32.714695 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b0d5f-08b8-4a21-9760-4634aa6f0ea5 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:06:32.720718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763eac1-3be6-4a75-91f0-6d59597ffadb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:06:32.721723 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763eac1-3be6-4a75-91f0-6d59597ffadb service placement] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:32.722004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763eac1-3be6-4a75-91f0-6d59597ffadb service placement] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:32.722286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763eac1-3be6-4a75-91f0-6d59597ffadb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:32.731014 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763eac1-3be6-4a75-91f0-6d59597ffadb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:06:32.732638 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763eac1-3be6-4a75-91f0-6d59597ffadb service placement] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:06:32.742578 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 754/1521] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:06:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:06:39.444199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa772af-3582-48c7-bf76-cd1cccb4f7b9 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:39.444762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa772af-3582-48c7-bf76-cd1cccb4f7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:39.444762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa772af-3582-48c7-bf76-cd1cccb4f7b9 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:39.446128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 10:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:06:39.451476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f911d01-3edc-4d3a-9216-7ed8378155d8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:39.451937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f911d01-3edc-4d3a-9216-7ed8378155d8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:39.452040 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f911d01-3edc-4d3a-9216-7ed8378155d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:39.478293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4f911d01-3edc-4d3a-9216-7ed8378155d8 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:06:39.479524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4f911d01-3edc-4d3a-9216-7ed8378155d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:39.488154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 10:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:06:39.645901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa772af-3582-48c7-bf76-cd1cccb4f7b9 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:06:39.661564 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa772af-3582-48c7-bf76-cd1cccb4f7b9 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:06:39.667155 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f2024-8461-48e5-a2f8-74ccdbb7f202 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:06:39.667806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f2024-8461-48e5-a2f8-74ccdbb7f202 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:39.668088 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f2024-8461-48e5-a2f8-74ccdbb7f202 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:39.668348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f2024-8461-48e5-a2f8-74ccdbb7f202 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:39.676453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f2024-8461-48e5-a2f8-74ccdbb7f202 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:06:39.678489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f2024-8461-48e5-a2f8-74ccdbb7f202 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:06:39.688146 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 769/1524] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:06:40.501677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdee75-dc4a-47fc-8827-d53d6b98e221 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:40.502252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdee75-dc4a-47fc-8827-d53d6b98e221 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:40.502252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdee75-dc4a-47fc-8827-d53d6b98e221 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:40.503503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 10:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:06:40.509206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0cfe6-c215-4048-bcaf-63c196d36586 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:40.509836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0cfe6-c215-4048-bcaf-63c196d36586 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:40.510068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0cfe6-c215-4048-bcaf-63c196d36586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:40.529968 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-02b0cfe6-c215-4048-bcaf-63c196d36586 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:06:40.531199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-02b0cfe6-c215-4048-bcaf-63c196d36586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:40.540798 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 10:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:06:40.694130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdee75-dc4a-47fc-8827-d53d6b98e221 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:06:40.716113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdee75-dc4a-47fc-8827-d53d6b98e221 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:06:40.723470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48400d74-61f8-4552-8c99-1ba628264175 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:06:40.724118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400d74-61f8-4552-8c99-1ba628264175 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:40.724433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400d74-61f8-4552-8c99-1ba628264175 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:40.724855 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400d74-61f8-4552-8c99-1ba628264175 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:40.734114 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48400d74-61f8-4552-8c99-1ba628264175 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:06:40.735707 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48400d74-61f8-4552-8c99-1ba628264175 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:06:40.745410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 757/1527] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:06:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:06:44.810982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340c00c-4340-420e-834d-743cc619370a None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:44.811847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340c00c-4340-420e-834d-743cc619370a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:44.811847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340c00c-4340-420e-834d-743cc619370a None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:44.813972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:06:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:06:44.819390 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c1a26-1950-4b0b-b094-6854e90a8d61 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:44.820394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c1a26-1950-4b0b-b094-6854e90a8d61 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:44.820394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c1a26-1950-4b0b-b094-6854e90a8d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:44.838402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ad4c1a26-1950-4b0b-b094-6854e90a8d61 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:06:44.839596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ad4c1a26-1950-4b0b-b094-6854e90a8d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:44.848417 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:06:44.921922 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340c00c-4340-420e-834d-743cc619370a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:06:44.924214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a340c00c-4340-420e-834d-743cc619370a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:44.989019 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340c00c-4340-420e-834d-743cc619370a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:06:44.993126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3945c-b1cb-413f-b43d-d8787ee57445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:06:44.993823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3945c-b1cb-413f-b43d-d8787ee57445 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:44.994111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3945c-b1cb-413f-b43d-d8787ee57445 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:44.994397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3945c-b1cb-413f-b43d-d8787ee57445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:45.003647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3945c-b1cb-413f-b43d-d8787ee57445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:06:45.005082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3945c-b1cb-413f-b43d-d8787ee57445 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:06:45.010057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:06:45.015003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 772/1530] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:06:47.085000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e95fd7-8294-4379-b994-27ff3fffb8eb None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:47.085589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e95fd7-8294-4379-b994-27ff3fffb8eb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:47.085589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e95fd7-8294-4379-b994-27ff3fffb8eb None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:47.087016 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:06:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:06:47.095422 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf0f46-d996-406c-a078-559de49968df None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:47.096486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf0f46-d996-406c-a078-559de49968df None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:47.096649 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf0f46-d996-406c-a078-559de49968df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:47.123391 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-87bf0f46-d996-406c-a078-559de49968df None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:06:47.124614 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-87bf0f46-d996-406c-a078-559de49968df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:47.134514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:06:47.174332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e95fd7-8294-4379-b994-27ff3fffb8eb None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:06:47.188542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e95fd7-8294-4379-b994-27ff3fffb8eb None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:06:47.192926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161aa6e-351d-4982-99b6-01d4d91377f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:06:47.193568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161aa6e-351d-4982-99b6-01d4d91377f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:47.193875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161aa6e-351d-4982-99b6-01d4d91377f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:47.194155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161aa6e-351d-4982-99b6-01d4d91377f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:47.203748 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161aa6e-351d-4982-99b6-01d4d91377f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:06:47.205211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161aa6e-351d-4982-99b6-01d4d91377f8 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:06:47.214796 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 760/1533] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:06:50.683725 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594f8bd-bd84-4fe2-bbb1-cdfdf5be28a7 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:50.684548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594f8bd-bd84-4fe2-bbb1-cdfdf5be28a7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:50.684733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594f8bd-bd84-4fe2-bbb1-cdfdf5be28a7 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:50.686924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:06:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:06:50.695621 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552b530-577e-4fb4-9130-036c92a67ac9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:50.696500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552b530-577e-4fb4-9130-036c92a67ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:50.696500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552b530-577e-4fb4-9130-036c92a67ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:50.703847 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=62 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:06:50.718642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d552b530-577e-4fb4-9130-036c92a67ac9 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:06:50.719874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d552b530-577e-4fb4-9130-036c92a67ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:50.729523 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:06:51.031081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594f8bd-bd84-4fe2-bbb1-cdfdf5be28a7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:06:51.033316 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8594f8bd-bd84-4fe2-bbb1-cdfdf5be28a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:51.095682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594f8bd-bd84-4fe2-bbb1-cdfdf5be28a7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:06:51.100453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af1f52-fa7c-4b0b-826f-31e5c3772be6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:06:51.101126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af1f52-fa7c-4b0b-826f-31e5c3772be6 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:51.101408 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af1f52-fa7c-4b0b-826f-31e5c3772be6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:51.101695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af1f52-fa7c-4b0b-826f-31e5c3772be6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:51.111473 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af1f52-fa7c-4b0b-826f-31e5c3772be6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:06:51.112923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af1f52-fa7c-4b0b-826f-31e5c3772be6 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:06:51.122771 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 775/1536] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:06:53.164313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad87294-9376-4cc1-9cf4-c47bfefb98a3 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:53.164898 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad87294-9376-4cc1-9cf4-c47bfefb98a3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:53.165393 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad87294-9376-4cc1-9cf4-c47bfefb98a3 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:53.166864 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 10:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:06:53.172424 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9c645-fc92-43b3-b3f9-b10da3dac520 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:53.172895 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9c645-fc92-43b3-b3f9-b10da3dac520 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:53.173001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9c645-fc92-43b3-b3f9-b10da3dac520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:53.191553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-90c9c645-fc92-43b3-b3f9-b10da3dac520 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:06:53.192750 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-90c9c645-fc92-43b3-b3f9-b10da3dac520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:53.201625 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 10:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:06:53.513475 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad87294-9376-4cc1-9cf4-c47bfefb98a3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:06:53.515457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8ad87294-9376-4cc1-9cf4-c47bfefb98a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:06:53.530873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad87294-9376-4cc1-9cf4-c47bfefb98a3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:06:53.534997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb86ba-fd1a-446b-99e2-feec98c55fd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:06:53.535633 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb86ba-fd1a-446b-99e2-feec98c55fd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:06:53.535921 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb86ba-fd1a-446b-99e2-feec98c55fd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:06:53.536193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb86ba-fd1a-446b-99e2-feec98c55fd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:06:53.546043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb86ba-fd1a-446b-99e2-feec98c55fd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:06:53.547487 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb86ba-fd1a-446b-99e2-feec98c55fd5 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:06:53.557736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 763/1539] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:07:03.186314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=32 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:07:03.552978 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=8 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:22.214492 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097f855-0476-492a-b36c-30f5783812c7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.215188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097f855-0476-492a-b36c-30f5783812c7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.215188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097f855-0476-492a-b36c-30f5783812c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.238961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a097f855-0476-492a-b36c-30f5783812c7 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.240586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a097f855-0476-492a-b36c-30f5783812c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.251061 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 777/1540] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.261290 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ede2c-a951-43e8-8707-9fdebd4a548a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.261933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ede2c-a951-43e8-8707-9fdebd4a548a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.261933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ede2c-a951-43e8-8707-9fdebd4a548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.270897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8baa78-7b02-48e3-a8c7-bde31757508c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.271615 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8baa78-7b02-48e3-a8c7-bde31757508c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.272123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8baa78-7b02-48e3-a8c7-bde31757508c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.290884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-369ede2c-a951-43e8-8707-9fdebd4a548a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.294126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-369ede2c-a951-43e8-8707-9fdebd4a548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.304491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0e8baa78-7b02-48e3-a8c7-bde31757508c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.306050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0e8baa78-7b02-48e3-a8c7-bde31757508c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.310593 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 764/1541] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.319331 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ede2c-a951-43e8-8707-9fdebd4a548a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:22.320853 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 778/1542] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:08:22.329369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892592e-9ddb-448d-9775-59e977828dca None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.330049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892592e-9ddb-448d-9775-59e977828dca None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.330550 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892592e-9ddb-448d-9775-59e977828dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.336208 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ede2c-a951-43e8-8707-9fdebd4a548a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:22.352366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f167602-118a-49b8-b120-88b35606bc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:22.353048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f167602-118a-49b8-b120-88b35606bc9c admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.353335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f167602-118a-49b8-b120-88b35606bc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.353611 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f167602-118a-49b8-b120-88b35606bc9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.354595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f167602-118a-49b8-b120-88b35606bc9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:08:22.354902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f167602-118a-49b8-b120-88b35606bc9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:22.356128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-6f167602-118a-49b8-b120-88b35606bc9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:08:22.356128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jul 04 10:08:22.356128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: Jul 04 10:08:22.357200 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f167602-118a-49b8-b120-88b35606bc9c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:22.366434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2892592e-9ddb-448d-9775-59e977828dca None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.367962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2892592e-9ddb-448d-9775-59e977828dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.379746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 765/1543] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 10:08:22.382993 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 779/1544] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.387632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f167602-118a-49b8-b120-88b35606bc9c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:22.391291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892592e-9ddb-448d-9775-59e977828dca None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:22.402911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f167602-118a-49b8-b120-88b35606bc9c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:22.408482 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892592e-9ddb-448d-9775-59e977828dca None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:22.411141 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83f29a-1f22-4425-812d-9b9e67aee75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:22.411924 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83f29a-1f22-4425-812d-9b9e67aee75e admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.412211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83f29a-1f22-4425-812d-9b9e67aee75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.412495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83f29a-1f22-4425-812d-9b9e67aee75e admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.413225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83f29a-1f22-4425-812d-9b9e67aee75e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:08:22.413535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83f29a-1f22-4425-812d-9b9e67aee75e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:22.415407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83f29a-1f22-4425-812d-9b9e67aee75e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:22.422271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fc772-811e-4241-b061-a04ab11507ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:22.423353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fc772-811e-4241-b061-a04ab11507ae admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.423865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fc772-811e-4241-b061-a04ab11507ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.424412 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fc772-811e-4241-b061-a04ab11507ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.425454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fc772-811e-4241-b061-a04ab11507ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:22.426793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-e39fc772-811e-4241-b061-a04ab11507ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:08:22.426793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jul 04 10:08:22.426793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: Jul 04 10:08:22.428535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fc772-811e-4241-b061-a04ab11507ae admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:22.437766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 766/1545] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:08:22.447534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83f29a-1f22-4425-812d-9b9e67aee75e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:22.467252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83f29a-1f22-4425-812d-9b9e67aee75e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:22.482281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 780/1546] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:08:22.487085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fc772-811e-4241-b061-a04ab11507ae admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:22.487427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47431e-8c77-4a5a-b052-8048e566837a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:22.488334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47431e-8c77-4a5a-b052-8048e566837a admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.488624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47431e-8c77-4a5a-b052-8048e566837a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.488901 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47431e-8c77-4a5a-b052-8048e566837a admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.489732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47431e-8c77-4a5a-b052-8048e566837a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:22.491874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47431e-8c77-4a5a-b052-8048e566837a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:22.552071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 767/1547] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/projects => generated 405 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:08:22.553467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602c33-ff20-43e4-9048-116c04116abb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.553749 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602c33-ff20-43e4-9048-116c04116abb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.554190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602c33-ff20-43e4-9048-116c04116abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.584683 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b4602c33-ff20-43e4-9048-116c04116abb None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.587276 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b4602c33-ff20-43e4-9048-116c04116abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.615520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fc772-811e-4241-b061-a04ab11507ae admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:22.688405 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 768/1548] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.689316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4602c33-ff20-43e4-9048-116c04116abb None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:22.702751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4602c33-ff20-43e4-9048-116c04116abb None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:22.704851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84541a71-f348-46ae-93c5-81681b497d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:22.705540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84541a71-f348-46ae-93c5-81681b497d8c admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.705865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84541a71-f348-46ae-93c5-81681b497d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.706141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84541a71-f348-46ae-93c5-81681b497d8c admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.707211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84541a71-f348-46ae-93c5-81681b497d8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:22.707742 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1188f-64dd-4dc6-bbf7-0fe7c0980fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:22.708416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1188f-64dd-4dc6-bbf7-0fe7c0980fdd admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.708701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1188f-64dd-4dc6-bbf7-0fe7c0980fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.708979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1188f-64dd-4dc6-bbf7-0fe7c0980fdd admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.709226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84541a71-f348-46ae-93c5-81681b497d8c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:22.709694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1188f-64dd-4dc6-bbf7-0fe7c0980fdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:22.710725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-def1188f-64dd-4dc6-bbf7-0fe7c0980fdd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:08:22.710725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jul 04 10:08:22.710725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: Jul 04 10:08:22.711938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1188f-64dd-4dc6-bbf7-0fe7c0980fdd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:22.730479 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 781/1549] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/users => generated 290 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:22.731864 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac5fab-871c-4011-8ec3-1a0d80576e19 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.732176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac5fab-871c-4011-8ec3-1a0d80576e19 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.732459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac5fab-871c-4011-8ec3-1a0d80576e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.739170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 769/1550] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/users => generated 303 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:22.739170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bcddde-a992-479b-878f-ed4a1b58abf2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.739170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bcddde-a992-479b-878f-ed4a1b58abf2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.739170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bcddde-a992-479b-878f-ed4a1b58abf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.753014 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-40ac5fab-871c-4011-8ec3-1a0d80576e19 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.754543 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-40ac5fab-871c-4011-8ec3-1a0d80576e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.756749 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-56bcddde-a992-479b-878f-ed4a1b58abf2 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.757968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-56bcddde-a992-479b-878f-ed4a1b58abf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.762693 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 782/1551] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.763877 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35a05e-f5e3-4750-873e-5eb79fc2b20c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.764152 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35a05e-f5e3-4750-873e-5eb79fc2b20c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.764433 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35a05e-f5e3-4750-873e-5eb79fc2b20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.766283 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 770/1552] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.771038 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd63dd4-98d1-4d83-94ef-b3a6a22e1268 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.771440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd63dd4-98d1-4d83-94ef-b3a6a22e1268 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.771839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd63dd4-98d1-4d83-94ef-b3a6a22e1268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.782583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3d35a05e-f5e3-4750-873e-5eb79fc2b20c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.783766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3d35a05e-f5e3-4750-873e-5eb79fc2b20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.790229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7bd63dd4-98d1-4d83-94ef-b3a6a22e1268 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.791437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7bd63dd4-98d1-4d83-94ef-b3a6a22e1268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.791621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 783/1553] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.797843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beb285-8a7c-4cb8-a396-cf8beab68956 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.797843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beb285-8a7c-4cb8-a396-cf8beab68956 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.797843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beb285-8a7c-4cb8-a396-cf8beab68956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.801212 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 771/1554] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.801992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32455ac-daee-412f-8eb3-c6d7d17e1381 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.802562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32455ac-daee-412f-8eb3-c6d7d17e1381 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.802899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32455ac-daee-412f-8eb3-c6d7d17e1381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.811495 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d5beb285-8a7c-4cb8-a396-cf8beab68956 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.812686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d5beb285-8a7c-4cb8-a396-cf8beab68956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.820694 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 784/1555] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.821263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c32455ac-daee-412f-8eb3-c6d7d17e1381 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.821474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5beb285-8a7c-4cb8-a396-cf8beab68956 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:22.822558 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c32455ac-daee-412f-8eb3-c6d7d17e1381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.830883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5beb285-8a7c-4cb8-a396-cf8beab68956 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:22.831978 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 772/1556] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.832837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32455ac-daee-412f-8eb3-c6d7d17e1381 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:22.835424 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238d38f-5375-45f6-a3d0-9404d235a128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:22.836079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238d38f-5375-45f6-a3d0-9404d235a128 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.836366 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238d38f-5375-45f6-a3d0-9404d235a128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.836644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238d38f-5375-45f6-a3d0-9404d235a128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.837360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238d38f-5375-45f6-a3d0-9404d235a128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:22.838997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238d38f-5375-45f6-a3d0-9404d235a128 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:22.843633 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32455ac-daee-412f-8eb3-c6d7d17e1381 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:22.848755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f439f-5b1c-45a0-abee-f034d8b95cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:22.852313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f439f-5b1c-45a0-abee-f034d8b95cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.852452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 785/1557] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:22.852548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f439f-5b1c-45a0-abee-f034d8b95cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.852548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f439f-5b1c-45a0-abee-f034d8b95cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.852726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfc03c-c70c-4adf-8785-dd392b9dc228 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.852816 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f439f-5b1c-45a0-abee-f034d8b95cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:22.852897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfc03c-c70c-4adf-8785-dd392b9dc228 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.852897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfc03c-c70c-4adf-8785-dd392b9dc228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.853062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f439f-5b1c-45a0-abee-f034d8b95cf5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:22.864809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 773/1558] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:22.864809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741e30-1f03-4a99-a90a-4cd5d655d8bd None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.864809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741e30-1f03-4a99-a90a-4cd5d655d8bd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.865194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741e30-1f03-4a99-a90a-4cd5d655d8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.872018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5edfc03c-c70c-4adf-8785-dd392b9dc228 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.873233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5edfc03c-c70c-4adf-8785-dd392b9dc228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.882869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 786/1559] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.883372 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2e741e30-1f03-4a99-a90a-4cd5d655d8bd None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.884113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e1296-ad87-4a82-9e1b-821439dcf510 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.884402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e1296-ad87-4a82-9e1b-821439dcf510 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.884680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e1296-ad87-4a82-9e1b-821439dcf510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.884884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2e741e30-1f03-4a99-a90a-4cd5d655d8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.893301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 774/1560] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.895518 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80b0a-c795-41b4-aa0c-be04eedeb4b7 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.895841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80b0a-c795-41b4-aa0c-be04eedeb4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.896137 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80b0a-c795-41b4-aa0c-be04eedeb4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.904679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-098e1296-ad87-4a82-9e1b-821439dcf510 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.905944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-098e1296-ad87-4a82-9e1b-821439dcf510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.914767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 787/1561] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.915687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-deb80b0a-c795-41b4-aa0c-be04eedeb4b7 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.915829 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e1296-ad87-4a82-9e1b-821439dcf510 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:22.916879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-deb80b0a-c795-41b4-aa0c-be04eedeb4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.928752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 775/1562] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.928934 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e1296-ad87-4a82-9e1b-821439dcf510 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:22.929039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f42c0-ddcb-4ca8-a560-be72ec0617a1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.929039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f42c0-ddcb-4ca8-a560-be72ec0617a1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.929039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f42c0-ddcb-4ca8-a560-be72ec0617a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.931399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e41d7f-b00a-4a2e-b0d4-20d21a764adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:22.932083 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e41d7f-b00a-4a2e-b0d4-20d21a764adf admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.932366 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e41d7f-b00a-4a2e-b0d4-20d21a764adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.932643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e41d7f-b00a-4a2e-b0d4-20d21a764adf admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.933370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e41d7f-b00a-4a2e-b0d4-20d21a764adf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:08:22.933673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e41d7f-b00a-4a2e-b0d4-20d21a764adf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:22.935293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e41d7f-b00a-4a2e-b0d4-20d21a764adf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:22.942648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:22.947298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1e1f42c0-ddcb-4ca8-a560-be72ec0617a1 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:22.955445 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 788/1563] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:08:22.955606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1e1f42c0-ddcb-4ca8-a560-be72ec0617a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:22.955716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e41d7f-b00a-4a2e-b0d4-20d21a764adf admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:22.959785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e41d7f-b00a-4a2e-b0d4-20d21a764adf admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:22.961597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 776/1564] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:22.965148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f42c0-ddcb-4ca8-a560-be72ec0617a1 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:22.965255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5e259-7591-45c6-accc-f4bea3108830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:22.965487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5e259-7591-45c6-accc-f4bea3108830 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.965817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5e259-7591-45c6-accc-f4bea3108830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.966089 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5e259-7591-45c6-accc-f4bea3108830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.966817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5e259-7591-45c6-accc-f4bea3108830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:22.968930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5e259-7591-45c6-accc-f4bea3108830 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:22.972494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f42c0-ddcb-4ca8-a560-be72ec0617a1 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:22.977130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fc2fb-5fdb-472c-9b25-986439bf65c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:22.977813 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fc2fb-5fdb-472c-9b25-986439bf65c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:22.978093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fc2fb-5fdb-472c-9b25-986439bf65c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:22.978364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fc2fb-5fdb-472c-9b25-986439bf65c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:22.979089 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fc2fb-5fdb-472c-9b25-986439bf65c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:22.983427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 789/1565] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:22.983596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fc2fb-5fdb-472c-9b25-986439bf65c4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:22.983708 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5e259-7591-45c6-accc-f4bea3108830 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:22.990784 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 777/1566] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:22.991581 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fc2fb-5fdb-472c-9b25-986439bf65c4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:22.993842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5e259-7591-45c6-accc-f4bea3108830 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:22.999644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c1725-b47f-4cd5-bf7a-4141dce7bf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.000666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c1725-b47f-4cd5-bf7a-4141dce7bf6b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.001035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c1725-b47f-4cd5-bf7a-4141dce7bf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.001431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c1725-b47f-4cd5-bf7a-4141dce7bf6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.001993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fc2fb-5fdb-472c-9b25-986439bf65c4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.002628 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c1725-b47f-4cd5-bf7a-4141dce7bf6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:08:23.003016 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c1725-b47f-4cd5-bf7a-4141dce7bf6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.004785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c1725-b47f-4cd5-bf7a-4141dce7bf6b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.008105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed3cb9-65f2-43a4-8a9d-ba9c4b2e7a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.008752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed3cb9-65f2-43a4-8a9d-ba9c4b2e7a85 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.009035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed3cb9-65f2-43a4-8a9d-ba9c4b2e7a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.009309 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed3cb9-65f2-43a4-8a9d-ba9c4b2e7a85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.010061 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed3cb9-65f2-43a4-8a9d-ba9c4b2e7a85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:08:23.010367 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed3cb9-65f2-43a4-8a9d-ba9c4b2e7a85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.011963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed3cb9-65f2-43a4-8a9d-ba9c4b2e7a85 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.016897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 790/1567] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:08:23.017561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c1725-b47f-4cd5-bf7a-4141dce7bf6b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.027318 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c1725-b47f-4cd5-bf7a-4141dce7bf6b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.033729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 778/1568] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:08:23.033729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed3cb9-65f2-43a4-8a9d-ba9c4b2e7a85 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.036943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517e459-d036-44c9-9e89-e5b60cf38fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.037722 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517e459-d036-44c9-9e89-e5b60cf38fae admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.038057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517e459-d036-44c9-9e89-e5b60cf38fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.038407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517e459-d036-44c9-9e89-e5b60cf38fae admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.039134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517e459-d036-44c9-9e89-e5b60cf38fae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:08:23.039441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517e459-d036-44c9-9e89-e5b60cf38fae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.041050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517e459-d036-44c9-9e89-e5b60cf38fae admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.044048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed3cb9-65f2-43a4-8a9d-ba9c4b2e7a85 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.048896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ed6a6-9c36-4eca-b4fa-d22d7f8209d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.049543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ed6a6-9c36-4eca-b4fa-d22d7f8209d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.049866 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ed6a6-9c36-4eca-b4fa-d22d7f8209d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.050145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ed6a6-9c36-4eca-b4fa-d22d7f8209d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.050937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ed6a6-9c36-4eca-b4fa-d22d7f8209d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:08:23.051246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ed6a6-9c36-4eca-b4fa-d22d7f8209d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.052838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ed6a6-9c36-4eca-b4fa-d22d7f8209d7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.053268 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 791/1569] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:08:23.058456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517e459-d036-44c9-9e89-e5b60cf38fae admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.061327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:23.064985 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 779/1570] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:08:23.065755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ed6a6-9c36-4eca-b4fa-d22d7f8209d7 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.074923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517e459-d036-44c9-9e89-e5b60cf38fae admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.076275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ed6a6-9c36-4eca-b4fa-d22d7f8209d7 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.079757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903606a-cf32-4c23-a0f4-b45c11f79d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.080417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903606a-cf32-4c23-a0f4-b45c11f79d42 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.080681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc30507-02b5-46ac-9e99-b8d2910a65bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.080801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903606a-cf32-4c23-a0f4-b45c11f79d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.080968 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903606a-cf32-4c23-a0f4-b45c11f79d42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.081343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc30507-02b5-46ac-9e99-b8d2910a65bf admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.081651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc30507-02b5-46ac-9e99-b8d2910a65bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.081766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903606a-cf32-4c23-a0f4-b45c11f79d42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.081927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc30507-02b5-46ac-9e99-b8d2910a65bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.082637 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc30507-02b5-46ac-9e99-b8d2910a65bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:08:23.082939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc30507-02b5-46ac-9e99-b8d2910a65bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.083876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903606a-cf32-4c23-a0f4-b45c11f79d42 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.084644 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc30507-02b5-46ac-9e99-b8d2910a65bf admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.096669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 780/1571] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:08:23.097496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc30507-02b5-46ac-9e99-b8d2910a65bf admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.107288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc30507-02b5-46ac-9e99-b8d2910a65bf admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.113409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f2115-14c8-40e6-8f66-1702086706ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.114181 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f2115-14c8-40e6-8f66-1702086706ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.114481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f2115-14c8-40e6-8f66-1702086706ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.114759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f2115-14c8-40e6-8f66-1702086706ff admin admin] PATH_INFO: `/v3/projects/c5f718c1e8ae45dbb633172c0ba74c5e/users/64835c93d17543668a8039caeeaa393c/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.116820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 792/1572] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:23.117535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903606a-cf32-4c23-a0f4-b45c11f79d42 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.133525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f2115-14c8-40e6-8f66-1702086706ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f718c1e8ae45dbb633172c0ba74c5e, user_id=64835c93d17543668a8039caeeaa393c, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.133525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f2115-14c8-40e6-8f66-1702086706ff admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.149956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 781/1573] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:23 2022] PUT /identity/v3/projects/c5f718c1e8ae45dbb633172c0ba74c5e/users/64835c93d17543668a8039caeeaa393c/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:23.150747 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f2115-14c8-40e6-8f66-1702086706ff admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.188438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903606a-cf32-4c23-a0f4-b45c11f79d42 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.212655 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f2115-14c8-40e6-8f66-1702086706ff admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.217421 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e747e-5032-4c2e-a3e9-771a71f1d248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.218121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e747e-5032-4c2e-a3e9-771a71f1d248 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.218407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e747e-5032-4c2e-a3e9-771a71f1d248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.218691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e747e-5032-4c2e-a3e9-771a71f1d248 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.219431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e747e-5032-4c2e-a3e9-771a71f1d248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.222583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e747e-5032-4c2e-a3e9-771a71f1d248 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.247105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483fddc-9f15-446c-85c1-7992350e5dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.248027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483fddc-9f15-446c-85c1-7992350e5dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.248284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483fddc-9f15-446c-85c1-7992350e5dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.248558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483fddc-9f15-446c-85c1-7992350e5dd0 admin admin] PATH_INFO: `/v3/projects/448f0d3c51ad4785945698c31bf45ecb/users/451a736eb35d4a6198208ad22e71abc6/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.250088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 782/1574] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:23.250948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e747e-5032-4c2e-a3e9-771a71f1d248 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.263030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483fddc-9f15-446c-85c1-7992350e5dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448f0d3c51ad4785945698c31bf45ecb, user_id=451a736eb35d4a6198208ad22e71abc6, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.264077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-b483fddc-9f15-446c-85c1-7992350e5dd0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:08:23.264077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jul 04 10:08:23.264077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: Jul 04 10:08:23.266035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483fddc-9f15-446c-85c1-7992350e5dd0 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.281309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 793/1575] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:23 2022] PUT /identity/v3/projects/448f0d3c51ad4785945698c31bf45ecb/users/451a736eb35d4a6198208ad22e71abc6/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:23.285308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483fddc-9f15-446c-85c1-7992350e5dd0 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.316973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:23.319014 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e747e-5032-4c2e-a3e9-771a71f1d248 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.360096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483fddc-9f15-446c-85c1-7992350e5dd0 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.365048 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e724a2d-5430-4c54-912c-4c84577e6a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.365744 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e724a2d-5430-4c54-912c-4c84577e6a4b admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.366033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e724a2d-5430-4c54-912c-4c84577e6a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.366312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e724a2d-5430-4c54-912c-4c84577e6a4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.367011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e724a2d-5430-4c54-912c-4c84577e6a4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.368972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e724a2d-5430-4c54-912c-4c84577e6a4b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.374874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:23.383754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c4ac3e-d24d-48f8-b1a9-407c63095893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.384884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4ac3e-d24d-48f8-b1a9-407c63095893 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.385249 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4ac3e-d24d-48f8-b1a9-407c63095893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.385728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4ac3e-d24d-48f8-b1a9-407c63095893 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.386787 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c4ac3e-d24d-48f8-b1a9-407c63095893 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.389127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c4ac3e-d24d-48f8-b1a9-407c63095893 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.399691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 794/1576] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:23.400541 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e724a2d-5430-4c54-912c-4c84577e6a4b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.424222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 783/1577] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/projects => generated 393 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:23.424222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c4ac3e-d24d-48f8-b1a9-407c63095893 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.467153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e724a2d-5430-4c54-912c-4c84577e6a4b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.484393 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c4ac3e-d24d-48f8-b1a9-407c63095893 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.489161 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851ef1f-46b2-4ac4-a572-72463cd353d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.489857 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ef1f-46b2-4ac4-a572-72463cd353d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.490143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ef1f-46b2-4ac4-a572-72463cd353d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.490420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ef1f-46b2-4ac4-a572-72463cd353d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.491119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851ef1f-46b2-4ac4-a572-72463cd353d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.493047 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851ef1f-46b2-4ac4-a572-72463cd353d3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.522213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 784/1578] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:23.522213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851ef1f-46b2-4ac4-a572-72463cd353d3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.527323 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d946ebf-cc98-46b4-a612-4ea25efc68ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.528080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d946ebf-cc98-46b4-a612-4ea25efc68ce admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.528339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d946ebf-cc98-46b4-a612-4ea25efc68ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.528667 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d946ebf-cc98-46b4-a612-4ea25efc68ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.529368 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d946ebf-cc98-46b4-a612-4ea25efc68ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.531420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d946ebf-cc98-46b4-a612-4ea25efc68ce admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.561244 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 795/1579] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:23.562650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e3dfc-a80e-4332-93c0-b05991792318 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.562943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e3dfc-a80e-4332-93c0-b05991792318 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.563234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e3dfc-a80e-4332-93c0-b05991792318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.580880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-674e3dfc-a80e-4332-93c0-b05991792318 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:23.582383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-674e3dfc-a80e-4332-93c0-b05991792318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:23.590322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851ef1f-46b2-4ac4-a572-72463cd353d3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.627467 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:23.647547 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 796/1580] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:23.647756 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996041e-f154-4a82-a298-fae105b50a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.648345 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e3dfc-a80e-4332-93c0-b05991792318 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.648525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996041e-f154-4a82-a298-fae105b50a41 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.648866 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996041e-f154-4a82-a298-fae105b50a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.649198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996041e-f154-4a82-a298-fae105b50a41 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.650004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996041e-f154-4a82-a298-fae105b50a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.652347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996041e-f154-4a82-a298-fae105b50a41 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.657355 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e3dfc-a80e-4332-93c0-b05991792318 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.662995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297e4fd-ae75-4b22-b1c6-7876201cfcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.663662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297e4fd-ae75-4b22-b1c6-7876201cfcb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.663944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297e4fd-ae75-4b22-b1c6-7876201cfcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.664218 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297e4fd-ae75-4b22-b1c6-7876201cfcb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.664907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297e4fd-ae75-4b22-b1c6-7876201cfcb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.666902 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297e4fd-ae75-4b22-b1c6-7876201cfcb4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.676556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 785/1581] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:23.678262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2ced2-dc8c-4ca4-ad39-53b542280feb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.678557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2ced2-dc8c-4ca4-ad39-53b542280feb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.678839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2ced2-dc8c-4ca4-ad39-53b542280feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.684609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:23.692430 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 797/1582] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/users => generated 302 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:23.693346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297e4fd-ae75-4b22-b1c6-7876201cfcb4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.699074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-99f2ced2-dc8c-4ca4-ad39-53b542280feb None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:23.700728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-99f2ced2-dc8c-4ca4-ad39-53b542280feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:23.704701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297e4fd-ae75-4b22-b1c6-7876201cfcb4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.709423 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 786/1583] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:23.710404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca825346-f88e-4ebb-a501-77e8247b9b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.710540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2ced2-dc8c-4ca4-ad39-53b542280feb None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.711140 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca825346-f88e-4ebb-a501-77e8247b9b4b admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.711426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca825346-f88e-4ebb-a501-77e8247b9b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.711708 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca825346-f88e-4ebb-a501-77e8247b9b4b admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.712698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca825346-f88e-4ebb-a501-77e8247b9b4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.714755 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca825346-f88e-4ebb-a501-77e8247b9b4b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.722928 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2ced2-dc8c-4ca4-ad39-53b542280feb None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.728085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44137e-449e-40b0-9698-bea6745de321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.728822 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44137e-449e-40b0-9698-bea6745de321 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.729152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44137e-449e-40b0-9698-bea6745de321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.729470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44137e-449e-40b0-9698-bea6745de321 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.730220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44137e-449e-40b0-9698-bea6745de321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.732349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44137e-449e-40b0-9698-bea6745de321 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.739947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 798/1584] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/users => generated 295 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:23.739947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca825346-f88e-4ebb-a501-77e8247b9b4b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.748994 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca825346-f88e-4ebb-a501-77e8247b9b4b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.753780 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c180508-6423-4750-afc7-7a54b771fb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.754720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c180508-6423-4750-afc7-7a54b771fb93 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.754863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 787/1585] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:23.754961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c180508-6423-4750-afc7-7a54b771fb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.755193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c180508-6423-4750-afc7-7a54b771fb93 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.755565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44137e-449e-40b0-9698-bea6745de321 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.755913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c180508-6423-4750-afc7-7a54b771fb93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.758126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c180508-6423-4750-afc7-7a54b771fb93 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.765972 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44137e-449e-40b0-9698-bea6745de321 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.771811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95a523-3eaf-425c-bce7-1d3a4c961363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.772547 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95a523-3eaf-425c-bce7-1d3a4c961363 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.772898 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95a523-3eaf-425c-bce7-1d3a4c961363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.773180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95a523-3eaf-425c-bce7-1d3a4c961363 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.774023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95a523-3eaf-425c-bce7-1d3a4c961363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.775967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95a523-3eaf-425c-bce7-1d3a4c961363 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.780048 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 799/1586] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:23.781053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c180508-6423-4750-afc7-7a54b771fb93 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.798058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 788/1587] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:23 2022] POST /identity/v3/users => generated 288 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:23.799193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95a523-3eaf-425c-bce7-1d3a4c961363 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.810037 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95a523-3eaf-425c-bce7-1d3a4c961363 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.815045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa40da-ca51-4001-bd3b-05d6a7cf021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.815795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa40da-ca51-4001-bd3b-05d6a7cf021a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.816215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa40da-ca51-4001-bd3b-05d6a7cf021a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.816696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa40da-ca51-4001-bd3b-05d6a7cf021a admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.817429 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa40da-ca51-4001-bd3b-05d6a7cf021a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.819097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa40da-ca51-4001-bd3b-05d6a7cf021a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.830108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 789/1588] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:23.830912 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa40da-ca51-4001-bd3b-05d6a7cf021a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.841080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c180508-6423-4750-afc7-7a54b771fb93 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.842678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa40da-ca51-4001-bd3b-05d6a7cf021a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.844647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69870c6-17ac-4c9e-b627-a5ffd24cdca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.845681 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69870c6-17ac-4c9e-b627-a5ffd24cdca9 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.845971 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69870c6-17ac-4c9e-b627-a5ffd24cdca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.846254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69870c6-17ac-4c9e-b627-a5ffd24cdca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.848651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac687e09-1189-43e2-af3b-0e87ee1bad49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.849299 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac687e09-1189-43e2-af3b-0e87ee1bad49 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.849582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac687e09-1189-43e2-af3b-0e87ee1bad49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.849899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac687e09-1189-43e2-af3b-0e87ee1bad49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.850623 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac687e09-1189-43e2-af3b-0e87ee1bad49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.852217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac687e09-1189-43e2-af3b-0e87ee1bad49 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.855953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69870c6-17ac-4c9e-b627-a5ffd24cdca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.857777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69870c6-17ac-4c9e-b627-a5ffd24cdca9 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.862467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 790/1589] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:23.863459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac687e09-1189-43e2-af3b-0e87ee1bad49 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.865203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ac687e09-1189-43e2-af3b-0e87ee1bad49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:23.870246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 800/1590] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:23.871068 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69870c6-17ac-4c9e-b627-a5ffd24cdca9 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.877892 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac687e09-1189-43e2-af3b-0e87ee1bad49 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.880829 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69870c6-17ac-4c9e-b627-a5ffd24cdca9 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.881651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dbe97-ae6e-4a1b-a6b0-a89d430f9ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.882358 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dbe97-ae6e-4a1b-a6b0-a89d430f9ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.882790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dbe97-ae6e-4a1b-a6b0-a89d430f9ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.883204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dbe97-ae6e-4a1b-a6b0-a89d430f9ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.886601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ee548-d42b-46a7-a965-b11a5a594fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.887268 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ee548-d42b-46a7-a965-b11a5a594fbc admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.887590 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ee548-d42b-46a7-a965-b11a5a594fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.887845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ee548-d42b-46a7-a965-b11a5a594fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.888597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ee548-d42b-46a7-a965-b11a5a594fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.890240 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ee548-d42b-46a7-a965-b11a5a594fbc admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.894852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3dbe97-ae6e-4a1b-a6b0-a89d430f9ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.896556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3dbe97-ae6e-4a1b-a6b0-a89d430f9ca6 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.906887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 801/1591] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:23.907891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ee548-d42b-46a7-a965-b11a5a594fbc admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.908247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 791/1592] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:23.909082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dbe97-ae6e-4a1b-a6b0-a89d430f9ca6 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.919189 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dbe97-ae6e-4a1b-a6b0-a89d430f9ca6 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.929318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86751fca-2fb6-499c-a679-291a4703944c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.929923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ee548-d42b-46a7-a965-b11a5a594fbc admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.930077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86751fca-2fb6-499c-a679-291a4703944c admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.930389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86751fca-2fb6-499c-a679-291a4703944c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.930612 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86751fca-2fb6-499c-a679-291a4703944c admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.931510 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86751fca-2fb6-499c-a679-291a4703944c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.933185 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86751fca-2fb6-499c-a679-291a4703944c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.938904 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e6d73-281f-4104-ac0c-405eb5bae5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.939685 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e6d73-281f-4104-ac0c-405eb5bae5ef admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.940006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e6d73-281f-4104-ac0c-405eb5bae5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.940425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e6d73-281f-4104-ac0c-405eb5bae5ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.941210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e6d73-281f-4104-ac0c-405eb5bae5ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.943053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e6d73-281f-4104-ac0c-405eb5bae5ef admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.949850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 792/1593] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:23.952635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86751fca-2fb6-499c-a679-291a4703944c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.955635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 802/1594] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:23.956886 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e6d73-281f-4104-ac0c-405eb5bae5ef admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.964953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86751fca-2fb6-499c-a679-291a4703944c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.968912 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e6d73-281f-4104-ac0c-405eb5bae5ef admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:23.970830 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765ac07-b6fa-4182-92b4-6741132c6c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.971597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765ac07-b6fa-4182-92b4-6741132c6c54 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.971980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765ac07-b6fa-4182-92b4-6741132c6c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.972727 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765ac07-b6fa-4182-92b4-6741132c6c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.973215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765ac07-b6fa-4182-92b4-6741132c6c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.974943 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765ac07-b6fa-4182-92b4-6741132c6c54 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.975145 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807f8d8-c05b-4af1-b2b4-1b88fe5d4c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:23.975828 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807f8d8-c05b-4af1-b2b4-1b88fe5d4c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:23.976113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807f8d8-c05b-4af1-b2b4-1b88fe5d4c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:23.976394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807f8d8-c05b-4af1-b2b4-1b88fe5d4c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:23.977121 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807f8d8-c05b-4af1-b2b4-1b88fe5d4c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:23.978918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807f8d8-c05b-4af1-b2b4-1b88fe5d4c9b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:23.988410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 793/1595] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:23.989423 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765ac07-b6fa-4182-92b4-6741132c6c54 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:23.992879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 803/1596] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:23.992879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807f8d8-c05b-4af1-b2b4-1b88fe5d4c9b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:24.006565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765ac07-b6fa-4182-92b4-6741132c6c54 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:24.012973 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b987f1-6431-4b6d-945a-e744b6a726dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:24.013697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b987f1-6431-4b6d-945a-e744b6a726dc admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.014081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807f8d8-c05b-4af1-b2b4-1b88fe5d4c9b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:24.014207 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b987f1-6431-4b6d-945a-e744b6a726dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.014560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b987f1-6431-4b6d-945a-e744b6a726dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.015573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b987f1-6431-4b6d-945a-e744b6a726dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:24.017522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b987f1-6431-4b6d-945a-e744b6a726dc admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:24.020021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55170d9-f025-4073-abac-4110fd721c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:24.020897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55170d9-f025-4073-abac-4110fd721c54 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.021292 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55170d9-f025-4073-abac-4110fd721c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.021771 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55170d9-f025-4073-abac-4110fd721c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.022584 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55170d9-f025-4073-abac-4110fd721c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:24.024381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55170d9-f025-4073-abac-4110fd721c54 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:24.032584 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 794/1597] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:24.033990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b987f1-6431-4b6d-945a-e744b6a726dc admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:24.041225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 804/1598] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:24.042153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55170d9-f025-4073-abac-4110fd721c54 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:24.049426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b987f1-6431-4b6d-945a-e744b6a726dc admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:24.055343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a078be-5ccd-4fec-933c-d6c0c4330df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:24.055735 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55170d9-f025-4073-abac-4110fd721c54 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:24.056143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a078be-5ccd-4fec-933c-d6c0c4330df1 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.056473 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a078be-5ccd-4fec-933c-d6c0c4330df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.056860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a078be-5ccd-4fec-933c-d6c0c4330df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.057595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a078be-5ccd-4fec-933c-d6c0c4330df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:24.059266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a078be-5ccd-4fec-933c-d6c0c4330df1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:24.061576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95c3ba-6827-4b70-ac5c-02541544143a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:24.062789 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95c3ba-6827-4b70-ac5c-02541544143a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.063130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95c3ba-6827-4b70-ac5c-02541544143a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.063351 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95c3ba-6827-4b70-ac5c-02541544143a admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.064087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95c3ba-6827-4b70-ac5c-02541544143a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:24.067006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95c3ba-6827-4b70-ac5c-02541544143a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:24.076949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 795/1599] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:24.076949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a078be-5ccd-4fec-933c-d6c0c4330df1 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:24.091399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 805/1600] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:24.091399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95c3ba-6827-4b70-ac5c-02541544143a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:24.094157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a078be-5ccd-4fec-933c-d6c0c4330df1 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:24.101358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95c3ba-6827-4b70-ac5c-02541544143a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:24.101582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88c757-4cee-4857-afc8-9c240e36eefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:24.102329 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88c757-4cee-4857-afc8-9c240e36eefe admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.102596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88c757-4cee-4857-afc8-9c240e36eefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.103845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88c757-4cee-4857-afc8-9c240e36eefe admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.104726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88c757-4cee-4857-afc8-9c240e36eefe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:24.106297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15065876-1af5-4426-bb04-93f0fd6aa4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:24.107190 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15065876-1af5-4426-bb04-93f0fd6aa4fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.107556 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15065876-1af5-4426-bb04-93f0fd6aa4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.107815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15065876-1af5-4426-bb04-93f0fd6aa4fb admin admin] PATH_INFO: `/v3/projects/2f559cbed35f4fbb8fe71906696733e7/users/dfba262d439a4253bf1c2c0f114d0cdc/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.107945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88c757-4cee-4857-afc8-9c240e36eefe admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:24.126691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 796/1601] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:24.127552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88c757-4cee-4857-afc8-9c240e36eefe admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:24.133831 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15065876-1af5-4426-bb04-93f0fd6aa4fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f559cbed35f4fbb8fe71906696733e7, user_id=dfba262d439a4253bf1c2c0f114d0cdc, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:24.135031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15065876-1af5-4426-bb04-93f0fd6aa4fb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:24.147956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88c757-4cee-4857-afc8-9c240e36eefe admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:24.154324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 806/1602] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:24 2022] PUT /identity/v3/projects/2f559cbed35f4fbb8fe71906696733e7/users/dfba262d439a4253bf1c2c0f114d0cdc/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:24.155435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15065876-1af5-4426-bb04-93f0fd6aa4fb admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:24.188447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:24.255203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84662570-63a1-4493-bec5-75a3d60a3b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:24.256123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84662570-63a1-4493-bec5-75a3d60a3b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.256414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84662570-63a1-4493-bec5-75a3d60a3b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.256696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84662570-63a1-4493-bec5-75a3d60a3b52 admin admin] PATH_INFO: `/v3/projects/b89cc6b0d41a4d3ea5a4c6220fa74bd8/users/277baadaab6e46a6a92aac80c01c132e/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.275421 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84662570-63a1-4493-bec5-75a3d60a3b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b89cc6b0d41a4d3ea5a4c6220fa74bd8, user_id=277baadaab6e46a6a92aac80c01c132e, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:24.278677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84662570-63a1-4493-bec5-75a3d60a3b52 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:24.286129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15065876-1af5-4426-bb04-93f0fd6aa4fb admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:24.292679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86cb29-850f-452b-99f5-ef1d829d933c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:24.293946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86cb29-850f-452b-99f5-ef1d829d933c admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.294398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86cb29-850f-452b-99f5-ef1d829d933c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.294835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86cb29-850f-452b-99f5-ef1d829d933c admin admin] PATH_INFO: `/v3/projects/b852926ec18f4dc695da5ec1c5582f48/users/1c05b9b08df0437f9b0abb167d17aa56/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.307087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 797/1603] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:24 2022] PUT /identity/v3/projects/b89cc6b0d41a4d3ea5a4c6220fa74bd8/users/277baadaab6e46a6a92aac80c01c132e/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:24.308604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84662570-63a1-4493-bec5-75a3d60a3b52 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:24.326248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86cb29-850f-452b-99f5-ef1d829d933c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b852926ec18f4dc695da5ec1c5582f48, user_id=1c05b9b08df0437f9b0abb167d17aa56, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:24.329373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86cb29-850f-452b-99f5-ef1d829d933c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:24.342637 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:24.352963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 807/1604] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:24 2022] PUT /identity/v3/projects/b852926ec18f4dc695da5ec1c5582f48/users/1c05b9b08df0437f9b0abb167d17aa56/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:24.354146 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86cb29-850f-452b-99f5-ef1d829d933c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:24.435953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84662570-63a1-4493-bec5-75a3d60a3b52 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:24.451874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86cb29-850f-452b-99f5-ef1d829d933c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:24.463345 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af39ca1-8161-413d-97ca-f6e01037a626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:24.464885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af39ca1-8161-413d-97ca-f6e01037a626 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.465215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af39ca1-8161-413d-97ca-f6e01037a626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.465500 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af39ca1-8161-413d-97ca-f6e01037a626 admin admin] PATH_INFO: `/v3/projects/7f101e48ae9b4936a6d1b7bae000d7e1/users/fc9c26318d6c41f6b9455153f0d0e55b/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.487267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af39ca1-8161-413d-97ca-f6e01037a626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f101e48ae9b4936a6d1b7bae000d7e1, user_id=fc9c26318d6c41f6b9455153f0d0e55b, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:24.490122 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af39ca1-8161-413d-97ca-f6e01037a626 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:24.516466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 808/1605] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:24 2022] PUT /identity/v3/projects/7f101e48ae9b4936a6d1b7bae000d7e1/users/fc9c26318d6c41f6b9455153f0d0e55b/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:24.516466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af39ca1-8161-413d-97ca-f6e01037a626 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:24.521243 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b7b99-865b-47c5-a25f-daf300a978d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:24.523346 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7b99-865b-47c5-a25f-daf300a978d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.523768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7b99-865b-47c5-a25f-daf300a978d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.524353 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7b99-865b-47c5-a25f-daf300a978d4 admin admin] PATH_INFO: `/v3/projects/a57f93d66f534107a228d908afa64890/users/88d2351fc3b74ef5a530fded1be6b997/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.544908 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b7b99-865b-47c5-a25f-daf300a978d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a57f93d66f534107a228d908afa64890, user_id=88d2351fc3b74ef5a530fded1be6b997, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:24.547942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b7b99-865b-47c5-a25f-daf300a978d4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:24.565339 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 798/1606] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:24 2022] PUT /identity/v3/projects/a57f93d66f534107a228d908afa64890/users/88d2351fc3b74ef5a530fded1be6b997/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:24.567463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba7b40-bbb2-414e-b289-5ecdbac11867 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.567861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba7b40-bbb2-414e-b289-5ecdbac11867 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.568247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba7b40-bbb2-414e-b289-5ecdbac11867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.587097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-45ba7b40-bbb2-414e-b289-5ecdbac11867 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:24.588853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-45ba7b40-bbb2-414e-b289-5ecdbac11867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:24.599460 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af39ca1-8161-413d-97ca-f6e01037a626 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:24.632250 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:24.663454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 799/1607] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:24.664845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dd8db-ffb4-4c43-b913-ba9fb8e217fe None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.665138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dd8db-ffb4-4c43-b913-ba9fb8e217fe None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.665440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dd8db-ffb4-4c43-b913-ba9fb8e217fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.673935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969fb90-cb12-45be-bd6f-0fc70bdcbd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:24.674901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969fb90-cb12-45be-bd6f-0fc70bdcbd4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.675303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969fb90-cb12-45be-bd6f-0fc70bdcbd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.675587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969fb90-cb12-45be-bd6f-0fc70bdcbd4b admin admin] PATH_INFO: `/v3/projects/c79e08ac71944d41853bc2ac057a288a/users/6301fe8744584190b1720c0e42299b91/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.694395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c19dd8db-ffb4-4c43-b913-ba9fb8e217fe None None] MFA Rules not processed for user `277baadaab6e46a6a92aac80c01c132e`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:24.695991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c19dd8db-ffb4-4c43-b913-ba9fb8e217fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:24.696403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969fb90-cb12-45be-bd6f-0fc70bdcbd4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c79e08ac71944d41853bc2ac057a288a, user_id=6301fe8744584190b1720c0e42299b91, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:24.701095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969fb90-cb12-45be-bd6f-0fc70bdcbd4b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:24.732923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 809/1608] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:24 2022] PUT /identity/v3/projects/c79e08ac71944d41853bc2ac057a288a/users/6301fe8744584190b1720c0e42299b91/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:24.732923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c35e21-f168-4e41-9369-3f99fb825672 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.732923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c35e21-f168-4e41-9369-3f99fb825672 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.732923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c35e21-f168-4e41-9369-3f99fb825672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.735667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:24.746532 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-68c35e21-f168-4e41-9369-3f99fb825672 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:24.747797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-68c35e21-f168-4e41-9369-3f99fb825672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:24.781564 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 800/1609] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:24.783193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb80ce0-b567-485b-8026-423d4f27f3d1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.783584 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb80ce0-b567-485b-8026-423d4f27f3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.783950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb80ce0-b567-485b-8026-423d4f27f3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.808324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4cb80ce0-b567-485b-8026-423d4f27f3d1 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:24.810021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4cb80ce0-b567-485b-8026-423d4f27f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:24.837009 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 810/1610] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:24.838678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f610e26-a436-408d-88dd-1196fe0ad342 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.838961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f610e26-a436-408d-88dd-1196fe0ad342 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.839258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f610e26-a436-408d-88dd-1196fe0ad342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.859431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9f610e26-a436-408d-88dd-1196fe0ad342 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:24.860638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9f610e26-a436-408d-88dd-1196fe0ad342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:24.869332 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 811/1611] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:24.870185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f610e26-a436-408d-88dd-1196fe0ad342 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:24.898013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 801/1612] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:24.899925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8fe6b-aabc-4479-91ca-983e3886d13f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.899925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8fe6b-aabc-4479-91ca-983e3886d13f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.900148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8fe6b-aabc-4479-91ca-983e3886d13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.939572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f610e26-a436-408d-88dd-1196fe0ad342 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:24.940507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-66c8fe6b-aabc-4479-91ca-983e3886d13f None None] MFA Rules not processed for user `6301fe8744584190b1720c0e42299b91`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:24.942155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-66c8fe6b-aabc-4479-91ca-983e3886d13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:24.944429 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce866b51-f4da-4a3d-8076-c672a1e3d090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:24.945155 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce866b51-f4da-4a3d-8076-c672a1e3d090 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:24.945445 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce866b51-f4da-4a3d-8076-c672a1e3d090 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:24.945888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce866b51-f4da-4a3d-8076-c672a1e3d090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:24.962549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce866b51-f4da-4a3d-8076-c672a1e3d090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:24.964541 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce866b51-f4da-4a3d-8076-c672a1e3d090 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:24.983239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 812/1613] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:24.984328 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce866b51-f4da-4a3d-8076-c672a1e3d090 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:24.995000 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce866b51-f4da-4a3d-8076-c672a1e3d090 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:25.001405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c9c16-572d-4e53-9da5-c3da0bdade23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:25.002170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c9c16-572d-4e53-9da5-c3da0bdade23 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.002453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c9c16-572d-4e53-9da5-c3da0bdade23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.002729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c9c16-572d-4e53-9da5-c3da0bdade23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.003633 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c9c16-572d-4e53-9da5-c3da0bdade23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:25.005907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c9c16-572d-4e53-9da5-c3da0bdade23 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:25.043395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 813/1614] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:24 2022] POST /identity/v3/projects => generated 405 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:25.045147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c9c16-572d-4e53-9da5-c3da0bdade23 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:25.081644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:25.107620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 802/1615] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:25.111286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8fe6b-aabc-4479-91ca-983e3886d13f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:25.128486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c9c16-572d-4e53-9da5-c3da0bdade23 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:25.134110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8fe6b-aabc-4479-91ca-983e3886d13f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:25.137584 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3aaf7-2abd-468d-9805-f2aa01caa1a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:25.138285 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3aaf7-2abd-468d-9805-f2aa01caa1a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.138566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3aaf7-2abd-468d-9805-f2aa01caa1a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.138845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3aaf7-2abd-468d-9805-f2aa01caa1a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.142637 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe407e9-bee5-468c-a7ad-7aded87fe027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:25.143290 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe407e9-bee5-468c-a7ad-7aded87fe027 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.143914 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe407e9-bee5-468c-a7ad-7aded87fe027 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.144074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe407e9-bee5-468c-a7ad-7aded87fe027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.251963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe407e9-bee5-468c-a7ad-7aded87fe027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:25.254091 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe407e9-bee5-468c-a7ad-7aded87fe027 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:25.258915 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:25.267000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 803/1616] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:25.268320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe407e9-bee5-468c-a7ad-7aded87fe027 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:25.286796 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe407e9-bee5-468c-a7ad-7aded87fe027 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:25.292325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05539ba9-d672-44af-b907-f51958976e30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:25.292325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05539ba9-d672-44af-b907-f51958976e30 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.292325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05539ba9-d672-44af-b907-f51958976e30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.292731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05539ba9-d672-44af-b907-f51958976e30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.306097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05539ba9-d672-44af-b907-f51958976e30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:25.307859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05539ba9-d672-44af-b907-f51958976e30 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:25.319492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 804/1617] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:25.320989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cc50c-977b-48c6-9b4a-8c736af02362 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.321372 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cc50c-977b-48c6-9b4a-8c736af02362 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.321797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cc50c-977b-48c6-9b4a-8c736af02362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.342809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3aaf7-2abd-468d-9805-f2aa01caa1a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:25.344291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3aaf7-2abd-468d-9805-f2aa01caa1a9 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:25.359914 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e87cc50c-977b-48c6-9b4a-8c736af02362 None None] MFA Rules not processed for user `451a736eb35d4a6198208ad22e71abc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:25.361293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e87cc50c-977b-48c6-9b4a-8c736af02362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:25.364553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 814/1618] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:25.365292 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3aaf7-2abd-468d-9805-f2aa01caa1a9 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:25.384014 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3aaf7-2abd-468d-9805-f2aa01caa1a9 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:25.397342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f1863-d64a-471e-bd4b-52aa288dfe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:25.398284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f1863-d64a-471e-bd4b-52aa288dfe2c admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.398570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f1863-d64a-471e-bd4b-52aa288dfe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.398855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f1863-d64a-471e-bd4b-52aa288dfe2c admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.399568 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f1863-d64a-471e-bd4b-52aa288dfe2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:25.405291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f1863-d64a-471e-bd4b-52aa288dfe2c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:25.450352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 815/1619] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:25 2022] POST /identity/v3/users => generated 309 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:25.457038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a226756-68f9-431a-aa52-45c953dd7ff1 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.457038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a226756-68f9-431a-aa52-45c953dd7ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.457506 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a226756-68f9-431a-aa52-45c953dd7ff1 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.463622 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 816/1620] 10.222.0.79 () {54 vars in 961 bytes} [Mon Jul 4 10:08:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:08:25.464730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a226756-68f9-431a-aa52-45c953dd7ff1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:25.471808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 805/1621] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:25 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:25.473998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29932bec-e17f-4b94-bdad-5f5a0e781b82 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.474953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29932bec-e17f-4b94-bdad-5f5a0e781b82 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.474953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29932bec-e17f-4b94-bdad-5f5a0e781b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.494873 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a226756-68f9-431a-aa52-45c953dd7ff1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:25.499102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-29932bec-e17f-4b94-bdad-5f5a0e781b82 None None] MFA Rules not processed for user `ffc2409cb7fc4373a74ddfb48d501d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:25.500670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-29932bec-e17f-4b94-bdad-5f5a0e781b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:25.505641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1edea-6ed2-4056-92d1-ac41bd524fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:25.506078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1edea-6ed2-4056-92d1-ac41bd524fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.506366 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1edea-6ed2-4056-92d1-ac41bd524fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.507021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1edea-6ed2-4056-92d1-ac41bd524fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.507392 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1edea-6ed2-4056-92d1-ac41bd524fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:25.518570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1edea-6ed2-4056-92d1-ac41bd524fd9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:25.540787 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 817/1622] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:25.541895 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1edea-6ed2-4056-92d1-ac41bd524fd9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:25.554219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1edea-6ed2-4056-92d1-ac41bd524fd9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:25.559869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591aad1-4772-4b65-94bb-8d77401328cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:25.561008 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591aad1-4772-4b65-94bb-8d77401328cf admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.561008 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591aad1-4772-4b65-94bb-8d77401328cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.561242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591aad1-4772-4b65-94bb-8d77401328cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.562067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591aad1-4772-4b65-94bb-8d77401328cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:25.564329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591aad1-4772-4b65-94bb-8d77401328cf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:25.618785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 818/1623] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:25 2022] POST /identity/v3/projects => generated 405 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:25.620525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591aad1-4772-4b65-94bb-8d77401328cf admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:25.657874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 806/1624] 10.222.0.79 () {58 vars in 1058 bytes} [Mon Jul 4 10:08:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:25.659934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29932bec-e17f-4b94-bdad-5f5a0e781b82 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:25.689410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29932bec-e17f-4b94-bdad-5f5a0e781b82 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:25.714776 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591aad1-4772-4b65-94bb-8d77401328cf admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:25.731361 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58d4ce-2aad-4981-b651-66f45b1b204e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:25.732475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58d4ce-2aad-4981-b651-66f45b1b204e admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.732912 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58d4ce-2aad-4981-b651-66f45b1b204e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.733357 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58d4ce-2aad-4981-b651-66f45b1b204e admin admin] PATH_INFO: `/v3/projects/86dc6a166f9c440bbb03151e5260fb38/users/db4cbf59ad0b4cdba8382633125a2efb/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.762543 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58d4ce-2aad-4981-b651-66f45b1b204e admin admin] RBAC: Authorizing `identity:create_grant(project_id=86dc6a166f9c440bbb03151e5260fb38, user_id=db4cbf59ad0b4cdba8382633125a2efb, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:25.765418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58d4ce-2aad-4981-b651-66f45b1b204e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:25.780905 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebba14-bbe4-475e-8d4a-43b4bb433a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:25.781596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebba14-bbe4-475e-8d4a-43b4bb433a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.781899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebba14-bbe4-475e-8d4a-43b4bb433a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.782503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebba14-bbe4-475e-8d4a-43b4bb433a2a admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.782851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebba14-bbe4-475e-8d4a-43b4bb433a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:25.784869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebba14-bbe4-475e-8d4a-43b4bb433a2a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:25.789486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 819/1625] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:25 2022] PUT /identity/v3/projects/86dc6a166f9c440bbb03151e5260fb38/users/db4cbf59ad0b4cdba8382633125a2efb/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:25.790414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58d4ce-2aad-4981-b651-66f45b1b204e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:25.808639 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:25.812457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:25.831111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 807/1626] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:25 2022] POST /identity/v3/users => generated 309 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:25.832331 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebba14-bbe4-475e-8d4a-43b4bb433a2a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:25.856720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebba14-bbe4-475e-8d4a-43b4bb433a2a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:25.933204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58d4ce-2aad-4981-b651-66f45b1b204e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:25.962199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdeb40-58bf-404f-8aba-cca59f68ce85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:25.963176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdeb40-58bf-404f-8aba-cca59f68ce85 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.964504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdeb40-58bf-404f-8aba-cca59f68ce85 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.965876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdeb40-58bf-404f-8aba-cca59f68ce85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.966762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd56ee1-4b92-4046-b6f8-f4c33460e9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:25.967681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd56ee1-4b92-4046-b6f8-f4c33460e9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:25.967965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd56ee1-4b92-4046-b6f8-f4c33460e9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:25.968347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd56ee1-4b92-4046-b6f8-f4c33460e9a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:25.969415 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd56ee1-4b92-4046-b6f8-f4c33460e9a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:25.972219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd56ee1-4b92-4046-b6f8-f4c33460e9a7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:25.989316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 808/1627] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:25.990388 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd56ee1-4b92-4046-b6f8-f4c33460e9a7 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:26.001309 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd56ee1-4b92-4046-b6f8-f4c33460e9a7 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:26.008548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b3cbc-9392-4966-a0fa-02e3b5c1a185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:26.009515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b3cbc-9392-4966-a0fa-02e3b5c1a185 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:26.010491 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b3cbc-9392-4966-a0fa-02e3b5c1a185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:26.010491 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b3cbc-9392-4966-a0fa-02e3b5c1a185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:26.011316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b3cbc-9392-4966-a0fa-02e3b5c1a185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:26.013261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b3cbc-9392-4966-a0fa-02e3b5c1a185 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:26.029136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 809/1628] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:26.029136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b3cbc-9392-4966-a0fa-02e3b5c1a185 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:26.041176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b3cbc-9392-4966-a0fa-02e3b5c1a185 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:26.047104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e728b-c753-4ccf-80ec-0106c04518c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:26.048196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e728b-c753-4ccf-80ec-0106c04518c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:26.048791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e728b-c753-4ccf-80ec-0106c04518c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:26.049172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e728b-c753-4ccf-80ec-0106c04518c1 admin admin] PATH_INFO: `/v3/projects/86dc6a166f9c440bbb03151e5260fb38/users/db4cbf59ad0b4cdba8382633125a2efb/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:26.052140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e728b-c753-4ccf-80ec-0106c04518c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86dc6a166f9c440bbb03151e5260fb38, user_id=db4cbf59ad0b4cdba8382633125a2efb, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:26.056213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e728b-c753-4ccf-80ec-0106c04518c1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:26.082492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 810/1629] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:26 2022] PUT /identity/v3/projects/86dc6a166f9c440bbb03151e5260fb38/users/db4cbf59ad0b4cdba8382633125a2efb/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:26.083814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e728b-c753-4ccf-80ec-0106c04518c1 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:26.110665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdeb40-58bf-404f-8aba-cca59f68ce85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:26.113656 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdeb40-58bf-404f-8aba-cca59f68ce85 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:26.145970 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 820/1630] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:26.147596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c323e8-e1f1-4176-a5c9-ece716077d06 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:26.148214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c323e8-e1f1-4176-a5c9-ece716077d06 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:26.148646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c323e8-e1f1-4176-a5c9-ece716077d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:26.176922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e728b-c753-4ccf-80ec-0106c04518c1 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:26.193754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c7add-3a1d-4126-b65d-cbdc483743f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:26.194830 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c7add-3a1d-4126-b65d-cbdc483743f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:26.195218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c7add-3a1d-4126-b65d-cbdc483743f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:26.195504 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c7add-3a1d-4126-b65d-cbdc483743f9 admin admin] PATH_INFO: `/v3/projects/711e448fb91f424b84aa2834e3a4ff37/users/8cc0b86d074b47cab29497264d3d935d/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:26.205726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-46c323e8-e1f1-4176-a5c9-ece716077d06 None None] MFA Rules not processed for user `db4cbf59ad0b4cdba8382633125a2efb`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:26.207423 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-46c323e8-e1f1-4176-a5c9-ece716077d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:26.235961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c7add-3a1d-4126-b65d-cbdc483743f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=711e448fb91f424b84aa2834e3a4ff37, user_id=8cc0b86d074b47cab29497264d3d935d, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:26.240041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c7add-3a1d-4126-b65d-cbdc483743f9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:26.294647 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 811/1631] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:26 2022] PUT /identity/v3/projects/711e448fb91f424b84aa2834e3a4ff37/users/8cc0b86d074b47cab29497264d3d935d/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:26.296598 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c7add-3a1d-4126-b65d-cbdc483743f9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:26.371841 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 821/1632] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:26 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:26.383467 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c323e8-e1f1-4176-a5c9-ece716077d06 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:26.397957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:26.430799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c7add-3a1d-4126-b65d-cbdc483743f9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:26.443591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa55ab7-b045-4b1a-9273-79a17897485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:26.444517 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa55ab7-b045-4b1a-9273-79a17897485d admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:26.444663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa55ab7-b045-4b1a-9273-79a17897485d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:26.444871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa55ab7-b045-4b1a-9273-79a17897485d admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:26.445725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa55ab7-b045-4b1a-9273-79a17897485d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:26.447474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa55ab7-b045-4b1a-9273-79a17897485d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:26.459253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:26.474371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 812/1633] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:26.478171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa55ab7-b045-4b1a-9273-79a17897485d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:26.508041 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c323e8-e1f1-4176-a5c9-ece716077d06 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:26.525427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa55ab7-b045-4b1a-9273-79a17897485d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:26.546214 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8c487-60a8-48a4-9f2b-d7e7fc3af6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:26.547170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8c487-60a8-48a4-9f2b-d7e7fc3af6fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:26.547454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8c487-60a8-48a4-9f2b-d7e7fc3af6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:26.547736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8c487-60a8-48a4-9f2b-d7e7fc3af6fb admin admin] PATH_INFO: `/v3/projects/711e448fb91f424b84aa2834e3a4ff37/users/8cc0b86d074b47cab29497264d3d935d/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:26.559384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8c487-60a8-48a4-9f2b-d7e7fc3af6fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=711e448fb91f424b84aa2834e3a4ff37, user_id=8cc0b86d074b47cab29497264d3d935d, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:26.562152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8c487-60a8-48a4-9f2b-d7e7fc3af6fb admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:26.588252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 813/1634] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:26 2022] PUT /identity/v3/projects/711e448fb91f424b84aa2834e3a4ff37/users/8cc0b86d074b47cab29497264d3d935d/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:26.692535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fcf54-e521-4fa3-8a46-34d25ee515eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:26.693290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fcf54-e521-4fa3-8a46-34d25ee515eb service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:26.693670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fcf54-e521-4fa3-8a46-34d25ee515eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:26.694021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fcf54-e521-4fa3-8a46-34d25ee515eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:26.842641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fcf54-e521-4fa3-8a46-34d25ee515eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:26.842641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fcf54-e521-4fa3-8a46-34d25ee515eb service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:26.871505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 822/1635] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:26 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:08:26.975364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ce87e-0150-489d-8330-491df302b923 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:26.975915 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ce87e-0150-489d-8330-491df302b923 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:26.976339 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ce87e-0150-489d-8330-491df302b923 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:26.978240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 814/1636] 10.222.0.79 () {54 vars in 961 bytes} [Mon Jul 4 10:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:08:26.988964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7d62c-b034-43b0-a31f-8cb70b5d0903 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:26.989399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7d62c-b034-43b0-a31f-8cb70b5d0903 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:26.989958 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7d62c-b034-43b0-a31f-8cb70b5d0903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:27.023955 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-56d7d62c-b034-43b0-a31f-8cb70b5d0903 None None] MFA Rules not processed for user `ffc2409cb7fc4373a74ddfb48d501d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:27.025875 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-56d7d62c-b034-43b0-a31f-8cb70b5d0903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:27.179127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 823/1637] 10.222.0.79 () {58 vars in 1058 bytes} [Mon Jul 4 10:08:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:27.185588 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895f04f-771c-413d-b33f-9b53f1a736b5 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:27.190383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895f04f-771c-413d-b33f-9b53f1a736b5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:27.190850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895f04f-771c-413d-b33f-9b53f1a736b5 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:27.194225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 815/1638] 10.222.0.79 () {54 vars in 912 bytes} [Mon Jul 4 10:08:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:08:27.209852 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f47cc6-bc8f-4686-af7c-adfa160cf663 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:27.210178 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f47cc6-bc8f-4686-af7c-adfa160cf663 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:27.210467 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f47cc6-bc8f-4686-af7c-adfa160cf663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:27.258009 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-64f47cc6-bc8f-4686-af7c-adfa160cf663 None None] MFA Rules not processed for user `ffc2409cb7fc4373a74ddfb48d501d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:27.259292 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-64f47cc6-bc8f-4686-af7c-adfa160cf663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:27.280634 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 824/1639] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:08:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:27.297180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895f04f-771c-413d-b33f-9b53f1a736b5 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:27.318775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f47cc6-bc8f-4686-af7c-adfa160cf663 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:27.370527 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:27.446198 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f47cc6-bc8f-4686-af7c-adfa160cf663 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:27.455002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962538af-1018-4d49-9630-e9a2375db1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:27.457289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962538af-1018-4d49-9630-e9a2375db1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:27.458041 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962538af-1018-4d49-9630-e9a2375db1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:27.458442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962538af-1018-4d49-9630-e9a2375db1c5 admin admin] PATH_INFO: `/v3/users/277baadaab6e46a6a92aac80c01c132e` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:27.461159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962538af-1018-4d49-9630-e9a2375db1c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=277baadaab6e46a6a92aac80c01c132e)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:27.476251 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-962538af-1018-4d49-9630-e9a2375db1c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:08:27.476251 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jul 04 10:08:27.476251 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: Jul 04 10:08:27.480403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962538af-1018-4d49-9630-e9a2375db1c5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:27.484885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895f04f-771c-413d-b33f-9b53f1a736b5 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:27.497720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c8864-7dc7-4370-8eb1-f4ebddf15795 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:27.498411 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c8864-7dc7-4370-8eb1-f4ebddf15795 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:27.498697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c8864-7dc7-4370-8eb1-f4ebddf15795 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:27.498975 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c8864-7dc7-4370-8eb1-f4ebddf15795 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:27.521570 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c8864-7dc7-4370-8eb1-f4ebddf15795 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:27.523078 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c8864-7dc7-4370-8eb1-f4ebddf15795 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:27.549384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 816/1640] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:27.635066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-962538af-1018-4d49-9630-e9a2375db1c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '277baadaab6e46a6a92aac80c01c132e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:27.636893 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-962538af-1018-4d49-9630-e9a2375db1c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '277baadaab6e46a6a92aac80c01c132e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:27.661826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-962538af-1018-4d49-9630-e9a2375db1c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '277baadaab6e46a6a92aac80c01c132e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:27.702742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-962538af-1018-4d49-9630-e9a2375db1c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '277baadaab6e46a6a92aac80c01c132e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:27.704684 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-962538af-1018-4d49-9630-e9a2375db1c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '277baadaab6e46a6a92aac80c01c132e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:27.777553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05224f-c92a-4ae6-adec-9396788216d0 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:27.779110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05224f-c92a-4ae6-adec-9396788216d0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:27.779110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05224f-c92a-4ae6-adec-9396788216d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:27.783890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 825/1641] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:08:27 2022] DELETE /identity/v3/users/277baadaab6e46a6a92aac80c01c132e => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:27.814327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-8d05224f-c92a-4ae6-adec-9396788216d0 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:27.815758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8d05224f-c92a-4ae6-adec-9396788216d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:27.847029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d042176-4e27-44f3-a680-f0aa0295c97c None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:27.848586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d042176-4e27-44f3-a680-f0aa0295c97c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:27.849311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d042176-4e27-44f3-a680-f0aa0295c97c None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:27.851739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 826/1642] 10.222.0.79 () {54 vars in 912 bytes} [Mon Jul 4 10:08:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:08:27.858855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c9445-2e14-4fa4-a609-f8b624c7d9fe None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:27.860266 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c9445-2e14-4fa4-a609-f8b624c7d9fe None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:27.860953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c9445-2e14-4fa4-a609-f8b624c7d9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:27.895509 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a53c9445-2e14-4fa4-a609-f8b624c7d9fe None None] MFA Rules not processed for user `ffc2409cb7fc4373a74ddfb48d501d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:27.898631 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a53c9445-2e14-4fa4-a609-f8b624c7d9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:27.944238 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 817/1643] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:08:28.008378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05224f-c92a-4ae6-adec-9396788216d0 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:28.010221 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8d05224f-c92a-4ae6-adec-9396788216d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:28.035575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 827/1644] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:08:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:28.036841 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c9445-2e14-4fa4-a609-f8b624c7d9fe None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:28.038448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a53c9445-2e14-4fa4-a609-f8b624c7d9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:28.093377 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:28.123142 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c9445-2e14-4fa4-a609-f8b624c7d9fe None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:28.129786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05224f-c92a-4ae6-adec-9396788216d0 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:28.132027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e30199-7dcb-45eb-99d8-9bd6e2322c80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:28.133123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e30199-7dcb-45eb-99d8-9bd6e2322c80 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:28.133487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e30199-7dcb-45eb-99d8-9bd6e2322c80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:28.133931 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a509abb-28e2-48ba-ac5a-e2982085cf79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:28.134195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e30199-7dcb-45eb-99d8-9bd6e2322c80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:28.134596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a509abb-28e2-48ba-ac5a-e2982085cf79 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:28.134888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a509abb-28e2-48ba-ac5a-e2982085cf79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:28.135200 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a509abb-28e2-48ba-ac5a-e2982085cf79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:28.154017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e30199-7dcb-45eb-99d8-9bd6e2322c80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:28.155796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e30199-7dcb-45eb-99d8-9bd6e2322c80 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:28.161885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a509abb-28e2-48ba-ac5a-e2982085cf79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:28.163367 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a509abb-28e2-48ba-ac5a-e2982085cf79 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:28.184586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 828/1645] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:28.184769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 818/1646] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:08:28.357386 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a509abb-28e2-48ba-ac5a-e2982085cf79 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:28.357386 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2a509abb-28e2-48ba-ac5a-e2982085cf79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:28.369916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a509abb-28e2-48ba-ac5a-e2982085cf79 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:28.374843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:28.375733 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:28.376206 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:28.376669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] PATH_INFO: `/v3/users/6301fe8744584190b1720c0e42299b91` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:28.378163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6301fe8744584190b1720c0e42299b91)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:28.380862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:28.398991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:28.452300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6301fe8744584190b1720c0e42299b91'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:28.453231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6301fe8744584190b1720c0e42299b91'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:28.469506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6301fe8744584190b1720c0e42299b91'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:28.497586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6301fe8744584190b1720c0e42299b91'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:28.498620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6301fe8744584190b1720c0e42299b91'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:28.518356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 819/1647] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:08:28 2022] DELETE /identity/v3/users/6301fe8744584190b1720c0e42299b91 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:28.521557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa7d7e-ed12-4328-bc36-b32f20c0ce76 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:28.522569 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa7d7e-ed12-4328-bc36-b32f20c0ce76 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:28.523128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa7d7e-ed12-4328-bc36-b32f20c0ce76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:28.550981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-51aa7d7e-ed12-4328-bc36-b32f20c0ce76 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:28.553317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-51aa7d7e-ed12-4328-bc36-b32f20c0ce76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:28.677080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 829/1648] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:28.736336 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:28.740140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:28.852721 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d201f-d2df-4e08-accf-10a5bd760053 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:28.857673 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed3636-2448-41f9-bb04-f7b5fdcac131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:28.859490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed3636-2448-41f9-bb04-f7b5fdcac131 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:28.859490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed3636-2448-41f9-bb04-f7b5fdcac131 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:28.859490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed3636-2448-41f9-bb04-f7b5fdcac131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:28.881079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed3636-2448-41f9-bb04-f7b5fdcac131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:28.883213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed3636-2448-41f9-bb04-f7b5fdcac131 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:28.899172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 820/1649] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:29.884064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa7d7e-ed12-4328-bc36-b32f20c0ce76 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:29.886223 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-51aa7d7e-ed12-4328-bc36-b32f20c0ce76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:29.900995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa7d7e-ed12-4328-bc36-b32f20c0ce76 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:29.919601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:29.920854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:29.921908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:29.922453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36 admin admin] PATH_INFO: `/v3/projects/c79e08ac71944d41853bc2ac057a288a` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:29.924584 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c79e08ac71944d41853bc2ac057a288a)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:29.929054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:30.013344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c79e08ac71944d41853bc2ac057a288a', 'request_id': 'req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:30.034753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c79e08ac71944d41853bc2ac057a288a', 'request_id': 'req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:30.043327 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c79e08ac71944d41853bc2ac057a288a', 'request_id': 'req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:30.061454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36 admin admin] The token cache is being invalidate because project c79e08ac71944d41853bc2ac057a288a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:08:30.061937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:30.062879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-55c1fae8-9f48-4225-ba0b-2bb18f44fa36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:30.067858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 830/1650] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:08:29 2022] DELETE /identity/v3/projects/c79e08ac71944d41853bc2ac057a288a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:30.072358 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717953d-d9d5-45c4-80b8-176b69ed5601 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:30.072914 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717953d-d9d5-45c4-80b8-176b69ed5601 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:30.073164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717953d-d9d5-45c4-80b8-176b69ed5601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:30.093200 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-8717953d-d9d5-45c4-80b8-176b69ed5601 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:30.094861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8717953d-d9d5-45c4-80b8-176b69ed5601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:30.192928 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 821/1651] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:30.195591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecef80-5745-4318-92b2-3f62404eccbe None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:30.195885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecef80-5745-4318-92b2-3f62404eccbe None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:30.196168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecef80-5745-4318-92b2-3f62404eccbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:30.222397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5fecef80-5745-4318-92b2-3f62404eccbe None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:30.223671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5fecef80-5745-4318-92b2-3f62404eccbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:30.239973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 831/1652] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:30.240201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717953d-d9d5-45c4-80b8-176b69ed5601 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:30.305328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717953d-d9d5-45c4-80b8-176b69ed5601 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:30.305328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea5c93-6d03-4fbe-a67c-aca2c5db1f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:30.305328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea5c93-6d03-4fbe-a67c-aca2c5db1f1c admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:30.305328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea5c93-6d03-4fbe-a67c-aca2c5db1f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:30.308121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea5c93-6d03-4fbe-a67c-aca2c5db1f1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:30.308121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea5c93-6d03-4fbe-a67c-aca2c5db1f1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:08:30.308121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea5c93-6d03-4fbe-a67c-aca2c5db1f1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:30.308121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea5c93-6d03-4fbe-a67c-aca2c5db1f1c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:30.315241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 822/1653] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:08:30.321668 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecef80-5745-4318-92b2-3f62404eccbe None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:30.373753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecef80-5745-4318-92b2-3f62404eccbe None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:30.379583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf281356-2032-45d8-8ed3-087742650cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:30.380373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf281356-2032-45d8-8ed3-087742650cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:30.380703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf281356-2032-45d8-8ed3-087742650cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:30.380987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf281356-2032-45d8-8ed3-087742650cd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:30.381718 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf281356-2032-45d8-8ed3-087742650cd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:30.384849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf281356-2032-45d8-8ed3-087742650cd0 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:30.438370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 832/1654] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:30 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:30.441763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea5c93-6d03-4fbe-a67c-aca2c5db1f1c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:30.457920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea5c93-6d03-4fbe-a67c-aca2c5db1f1c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:30.503275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:30.619884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d70b1-8734-47e6-a135-a4d6157d1151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:30.620818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d70b1-8734-47e6-a135-a4d6157d1151 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:30.621270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d70b1-8734-47e6-a135-a4d6157d1151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:30.621967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d70b1-8734-47e6-a135-a4d6157d1151 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:30.623179 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d70b1-8734-47e6-a135-a4d6157d1151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:30.625579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d70b1-8734-47e6-a135-a4d6157d1151 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:30.718410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 823/1655] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:30 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:30.725374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf281356-2032-45d8-8ed3-087742650cd0 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:30.746588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf281356-2032-45d8-8ed3-087742650cd0 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:30.753957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d70b1-8734-47e6-a135-a4d6157d1151 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:30.756447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-555d70b1-8734-47e6-a135-a4d6157d1151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:30.766400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e9439-d5d2-4bdc-a35e-22016cec44f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:30.767224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9439-d5d2-4bdc-a35e-22016cec44f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:30.767847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9439-d5d2-4bdc-a35e-22016cec44f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:30.768334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9439-d5d2-4bdc-a35e-22016cec44f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:30.771118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e9439-d5d2-4bdc-a35e-22016cec44f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:30.778980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e9439-d5d2-4bdc-a35e-22016cec44f4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:30.779495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d70b1-8734-47e6-a135-a4d6157d1151 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:30.798967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a29661-5901-4a15-9057-32d7e6df94a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:30.799510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 833/1656] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:30.800293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a29661-5901-4a15-9057-32d7e6df94a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:30.800586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a29661-5901-4a15-9057-32d7e6df94a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:30.800864 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a29661-5901-4a15-9057-32d7e6df94a5 admin admin] PATH_INFO: `/v3/projects/b89cc6b0d41a4d3ea5a4c6220fa74bd8` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:30.803887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a29661-5901-4a15-9057-32d7e6df94a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b89cc6b0d41a4d3ea5a4c6220fa74bd8)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:30.805115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-d7a29661-5901-4a15-9057-32d7e6df94a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 10:08:30.805115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jul 04 10:08:30.805115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: Jul 04 10:08:30.805658 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e9439-d5d2-4bdc-a35e-22016cec44f4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:30.817716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a29661-5901-4a15-9057-32d7e6df94a5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:30.826945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e9439-d5d2-4bdc-a35e-22016cec44f4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:30.846018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de522a-ae23-41ea-909f-c5f5da2d14c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:30.846979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de522a-ae23-41ea-909f-c5f5da2d14c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:30.847401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de522a-ae23-41ea-909f-c5f5da2d14c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:30.847810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de522a-ae23-41ea-909f-c5f5da2d14c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:30.848682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de522a-ae23-41ea-909f-c5f5da2d14c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:30.850513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de522a-ae23-41ea-909f-c5f5da2d14c8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:30.886573 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 834/1657] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:30.890616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de522a-ae23-41ea-909f-c5f5da2d14c8 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:30.910783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de522a-ae23-41ea-909f-c5f5da2d14c8 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:30.928515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:30.934576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d7a29661-5901-4a15-9057-32d7e6df94a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b89cc6b0d41a4d3ea5a4c6220fa74bd8', 'request_id': 'req-d7a29661-5901-4a15-9057-32d7e6df94a5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:30.942216 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d7a29661-5901-4a15-9057-32d7e6df94a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b89cc6b0d41a4d3ea5a4c6220fa74bd8', 'request_id': 'req-d7a29661-5901-4a15-9057-32d7e6df94a5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:30.973496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d7a29661-5901-4a15-9057-32d7e6df94a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b89cc6b0d41a4d3ea5a4c6220fa74bd8', 'request_id': 'req-d7a29661-5901-4a15-9057-32d7e6df94a5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:31.006833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d7a29661-5901-4a15-9057-32d7e6df94a5 admin admin] The token cache is being invalidate because project b89cc6b0d41a4d3ea5a4c6220fa74bd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:08:31.007326 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d7a29661-5901-4a15-9057-32d7e6df94a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:31.008237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d7a29661-5901-4a15-9057-32d7e6df94a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:31.010870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 824/1658] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:08:30 2022] DELETE /identity/v3/projects/b89cc6b0d41a4d3ea5a4c6220fa74bd8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:31.012311 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cdcb7-8d94-4ab9-86b9-19a1079603fb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:31.012734 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cdcb7-8d94-4ab9-86b9-19a1079603fb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:31.013144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cdcb7-8d94-4ab9-86b9-19a1079603fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:31.043252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbdcd5-d7bd-4132-a51c-167c3a244fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:31.044584 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbdcd5-d7bd-4132-a51c-167c3a244fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:31.044946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbdcd5-d7bd-4132-a51c-167c3a244fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:31.048341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbdcd5-d7bd-4132-a51c-167c3a244fa3 admin admin] PATH_INFO: `/v3/projects/81e9459d637748ffb6161ea762d9e2c3/users/11dee66e9c30407c889e172c1036505d/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:31.074339 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d46cdcb7-8d94-4ab9-86b9-19a1079603fb None None] MFA Rules not processed for user `8cc0b86d074b47cab29497264d3d935d`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:31.076567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d46cdcb7-8d94-4ab9-86b9-19a1079603fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:31.089860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbdcd5-d7bd-4132-a51c-167c3a244fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e9459d637748ffb6161ea762d9e2c3, user_id=11dee66e9c30407c889e172c1036505d, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:31.093029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbdcd5-d7bd-4132-a51c-167c3a244fa3 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:31.116039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 835/1659] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:30 2022] PUT /identity/v3/projects/81e9459d637748ffb6161ea762d9e2c3/users/11dee66e9c30407c889e172c1036505d/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:08:31.134193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbdcd5-d7bd-4132-a51c-167c3a244fa3 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:31.142038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-02bbdcd5-d7bd-4132-a51c-167c3a244fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:31.243614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 825/1660] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:31 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:31.249734 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81238893-a40c-41ac-8d43-6846a811badc None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:31.250041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81238893-a40c-41ac-8d43-6846a811badc None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:31.252483 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81238893-a40c-41ac-8d43-6846a811badc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:31.300706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-81238893-a40c-41ac-8d43-6846a811badc None None] MFA Rules not processed for user `11dee66e9c30407c889e172c1036505d`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:31.302056 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-81238893-a40c-41ac-8d43-6846a811badc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:31.302730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbdcd5-d7bd-4132-a51c-167c3a244fa3 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:31.312776 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:31.312959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:31.313858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:31.314057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:31.314257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] PATH_INFO: `/v3/users/db4cbf59ad0b4cdba8382633125a2efb` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:31.319019 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] RBAC: Authorizing `identity:delete_user(user_id=db4cbf59ad0b4cdba8382633125a2efb)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:31.321981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:31.463207 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 826/1661] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:31.464371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db4cbf59ad0b4cdba8382633125a2efb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:31.469145 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db4cbf59ad0b4cdba8382633125a2efb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:31.471362 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81238893-a40c-41ac-8d43-6846a811badc None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:31.483669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-81238893-a40c-41ac-8d43-6846a811badc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:31.501255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db4cbf59ad0b4cdba8382633125a2efb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:31.516801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81238893-a40c-41ac-8d43-6846a811badc None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:31.526939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617820e7-3d08-4ea0-8760-d11776716f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:31.527604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db4cbf59ad0b4cdba8382633125a2efb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:31.528315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db4cbf59ad0b4cdba8382633125a2efb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:31.529936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617820e7-3d08-4ea0-8760-d11776716f8a admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:31.530223 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617820e7-3d08-4ea0-8760-d11776716f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:31.530499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617820e7-3d08-4ea0-8760-d11776716f8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:31.532541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617820e7-3d08-4ea0-8760-d11776716f8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:31.537328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617820e7-3d08-4ea0-8760-d11776716f8a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:31.557689 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 836/1662] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:08:31 2022] DELETE /identity/v3/users/db4cbf59ad0b4cdba8382633125a2efb => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:31.603255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 827/1663] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:31 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:31.607950 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:31.610039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:31.632232 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:31.731520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1015bd-020c-45d9-b33f-c4f2f87cd31a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:31.879311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403850d-9cb3-4b79-ac88-ab6429b01478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:31.880728 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403850d-9cb3-4b79-ac88-ab6429b01478 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:31.881169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403850d-9cb3-4b79-ac88-ab6429b01478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:31.881455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403850d-9cb3-4b79-ac88-ab6429b01478 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:31.882284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403850d-9cb3-4b79-ac88-ab6429b01478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:31.885607 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403850d-9cb3-4b79-ac88-ab6429b01478 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:31.938259 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617820e7-3d08-4ea0-8760-d11776716f8a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:31.953134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 837/1664] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:31 2022] POST /identity/v3/users => generated 299 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:31.955456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617820e7-3d08-4ea0-8760-d11776716f8a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:31.986698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0315f-f9f9-491c-9365-f4d834653c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:31.988873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0315f-f9f9-491c-9365-f4d834653c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:31.991251 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0315f-f9f9-491c-9365-f4d834653c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:31.991571 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0315f-f9f9-491c-9365-f4d834653c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:31.993284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0315f-f9f9-491c-9365-f4d834653c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:32.002762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0315f-f9f9-491c-9365-f4d834653c2c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:32.032226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403850d-9cb3-4b79-ac88-ab6429b01478 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:32.034045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 828/1665] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:32.048942 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403850d-9cb3-4b79-ac88-ab6429b01478 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:32.061854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda01df-828a-4b82-809c-4f21d746af6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:32.063017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda01df-828a-4b82-809c-4f21d746af6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:32.063434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda01df-828a-4b82-809c-4f21d746af6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:32.063990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda01df-828a-4b82-809c-4f21d746af6d admin admin] PATH_INFO: `/v3/projects/d7641a4e75fd407ab1718d3716b50b1e/users/0864360df3504c18b05b04e3bb30655c/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:32.093167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda01df-828a-4b82-809c-4f21d746af6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7641a4e75fd407ab1718d3716b50b1e, user_id=0864360df3504c18b05b04e3bb30655c, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:32.097110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda01df-828a-4b82-809c-4f21d746af6d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:32.150830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 838/1666] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:32 2022] PUT /identity/v3/projects/d7641a4e75fd407ab1718d3716b50b1e/users/0864360df3504c18b05b04e3bb30655c/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:32.151526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0315f-f9f9-491c-9365-f4d834653c2c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:32.165855 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0315f-f9f9-491c-9365-f4d834653c2c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:32.338979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18701b8-2b73-4ef2-8e21-406c969eb162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:32.339678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18701b8-2b73-4ef2-8e21-406c969eb162 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:32.339962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18701b8-2b73-4ef2-8e21-406c969eb162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:32.340236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18701b8-2b73-4ef2-8e21-406c969eb162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:32.340973 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18701b8-2b73-4ef2-8e21-406c969eb162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:32.354894 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18701b8-2b73-4ef2-8e21-406c969eb162 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:32.385360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 829/1667] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:08:32.387337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda01df-828a-4b82-809c-4f21d746af6d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:32.414711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda01df-828a-4b82-809c-4f21d746af6d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:32.436390 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb4330-7c53-44b6-bdcb-aa381f1eada8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:32.436777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb4330-7c53-44b6-bdcb-aa381f1eada8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:32.443103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb4330-7c53-44b6-bdcb-aa381f1eada8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:32.443510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb4330-7c53-44b6-bdcb-aa381f1eada8 admin admin] PATH_INFO: `/v3/projects/d7641a4e75fd407ab1718d3716b50b1e/users/0864360df3504c18b05b04e3bb30655c/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:32.446995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb4330-7c53-44b6-bdcb-aa381f1eada8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7641a4e75fd407ab1718d3716b50b1e, user_id=0864360df3504c18b05b04e3bb30655c, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:32.450447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb4330-7c53-44b6-bdcb-aa381f1eada8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:32.498658 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 839/1668] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:32 2022] PUT /identity/v3/projects/d7641a4e75fd407ab1718d3716b50b1e/users/0864360df3504c18b05b04e3bb30655c/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:32.511374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e815c18-6d97-4d03-ab64-bbfedb0b08f8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:32.511777 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e815c18-6d97-4d03-ab64-bbfedb0b08f8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:32.512627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e815c18-6d97-4d03-ab64-bbfedb0b08f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:32.567903 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7e815c18-6d97-4d03-ab64-bbfedb0b08f8 None None] MFA Rules not processed for user `0864360df3504c18b05b04e3bb30655c`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:32.569543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7e815c18-6d97-4d03-ab64-bbfedb0b08f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:32.692490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:32.702510 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 830/1669] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:32.717643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb4330-7c53-44b6-bdcb-aa381f1eada8 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:32.719625 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0adb4330-7c53-44b6-bdcb-aa381f1eada8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:32.811127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb4330-7c53-44b6-bdcb-aa381f1eada8 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:32.819257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433aa6a7-0542-4992-86b1-b9e054a829c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:32.820511 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aa6a7-0542-4992-86b1-b9e054a829c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:32.821028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aa6a7-0542-4992-86b1-b9e054a829c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:32.821466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aa6a7-0542-4992-86b1-b9e054a829c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:32.837439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433aa6a7-0542-4992-86b1-b9e054a829c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:32.839874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433aa6a7-0542-4992-86b1-b9e054a829c4 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:32.868002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 840/1670] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:32.919166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e815c18-6d97-4d03-ab64-bbfedb0b08f8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:33.068085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e815c18-6d97-4d03-ab64-bbfedb0b08f8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:33.074630 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:33.076081 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:33.076594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:33.077068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] PATH_INFO: `/v3/users/11dee66e9c30407c889e172c1036505d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:33.079102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11dee66e9c30407c889e172c1036505d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:33.084633 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:33.186239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11dee66e9c30407c889e172c1036505d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:33.188377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11dee66e9c30407c889e172c1036505d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:33.214339 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11dee66e9c30407c889e172c1036505d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:33.240087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11dee66e9c30407c889e172c1036505d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:33.241402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11dee66e9c30407c889e172c1036505d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:33.258912 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 831/1671] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:08:32 2022] DELETE /identity/v3/users/11dee66e9c30407c889e172c1036505d => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:08:33.263215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913caa2-9a1a-4677-acdf-a619cc3fe9d1 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:33.264142 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913caa2-9a1a-4677-acdf-a619cc3fe9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:33.264618 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913caa2-9a1a-4677-acdf-a619cc3fe9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:33.328552 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a913caa2-9a1a-4677-acdf-a619cc3fe9d1 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:33.330642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a913caa2-9a1a-4677-acdf-a619cc3fe9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:33.346156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:33.452006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 841/1672] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:08:33.555105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:33.558114 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:33.704459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c612f0-e9ba-4dcf-9bb7-1c99965f5633 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:33.709663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f69a54-a0ad-45eb-9324-ddbb45a33a90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:33.710403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f69a54-a0ad-45eb-9324-ddbb45a33a90 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:33.710696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f69a54-a0ad-45eb-9324-ddbb45a33a90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:33.710975 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f69a54-a0ad-45eb-9324-ddbb45a33a90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:33.736128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f69a54-a0ad-45eb-9324-ddbb45a33a90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:33.739583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f69a54-a0ad-45eb-9324-ddbb45a33a90 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:33.779888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 832/1673] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:34.390682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76322e-406e-49cb-9539-55720b8c9156 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:34.391254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76322e-406e-49cb-9539-55720b8c9156 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:34.391484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76322e-406e-49cb-9539-55720b8c9156 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:34.393536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 842/1674] 10.222.0.79 () {54 vars in 921 bytes} [Mon Jul 4 10:08:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:08:34.527924 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b82804-bc45-4d21-a570-ec587cc1e08b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:34.528504 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b82804-bc45-4d21-a570-ec587cc1e08b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:34.528906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b82804-bc45-4d21-a570-ec587cc1e08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:34.569262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-39b82804-bc45-4d21-a570-ec587cc1e08b None None] MFA Rules not processed for user `5dc48318ab7944c197bf5f7673511931`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:34.570957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-39b82804-bc45-4d21-a570-ec587cc1e08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:34.611172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:34.753773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 833/1675] 10.222.0.79 () {58 vars in 1018 bytes} [Mon Jul 4 10:08:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:34.861530 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a8071-b077-4677-b90e-7863b3941c3d None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:34.861968 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a8071-b077-4677-b90e-7863b3941c3d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:34.862381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a8071-b077-4677-b90e-7863b3941c3d None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:34.863931 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 843/1676] 10.222.0.79 () {54 vars in 960 bytes} [Mon Jul 4 10:08:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:08:34.887146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b30656-f419-427a-a1ac-4c510516b7f8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:34.887435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b30656-f419-427a-a1ac-4c510516b7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:34.887726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b30656-f419-427a-a1ac-4c510516b7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:34.924117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-99b30656-f419-427a-a1ac-4c510516b7f8 None None] MFA Rules not processed for user `5dc48318ab7944c197bf5f7673511931`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:34.927945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-99b30656-f419-427a-a1ac-4c510516b7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:34.950120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 834/1677] 10.222.0.79 () {58 vars in 1057 bytes} [Mon Jul 4 10:08:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:34.959564 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a8071-b077-4677-b90e-7863b3941c3d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:34.988832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a8071-b077-4677-b90e-7863b3941c3d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:35.010733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedeebe6-bc9b-4e9b-8074-1be2ae6a8262 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:35.012080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedeebe6-bc9b-4e9b-8074-1be2ae6a8262 service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:35.013238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedeebe6-bc9b-4e9b-8074-1be2ae6a8262 service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:35.013531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedeebe6-bc9b-4e9b-8074-1be2ae6a8262 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:35.041446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedeebe6-bc9b-4e9b-8074-1be2ae6a8262 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:35.043328 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedeebe6-bc9b-4e9b-8074-1be2ae6a8262 service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:35.070202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 844/1678] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:08:36.271078 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b30656-f419-427a-a1ac-4c510516b7f8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:36.272974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-99b30656-f419-427a-a1ac-4c510516b7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:36.311677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b30656-f419-427a-a1ac-4c510516b7f8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:36.329754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:36.330674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:36.331832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:36.332236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18 admin admin] PATH_INFO: `/v3/projects/86dc6a166f9c440bbb03151e5260fb38` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:36.337096 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86dc6a166f9c440bbb03151e5260fb38)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:36.346912 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:36.505760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86dc6a166f9c440bbb03151e5260fb38', 'request_id': 'req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:36.522596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86dc6a166f9c440bbb03151e5260fb38', 'request_id': 'req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:36.546522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86dc6a166f9c440bbb03151e5260fb38', 'request_id': 'req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:36.596972 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18 admin admin] The token cache is being invalidate because project 86dc6a166f9c440bbb03151e5260fb38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:08:36.597645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:36.599229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e39c7a17-df3e-4ce5-a4b7-1fa2adc5cd18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:36.605524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 835/1679] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:08:36 2022] DELETE /identity/v3/projects/86dc6a166f9c440bbb03151e5260fb38 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:36.610428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e249e3-796a-4433-b502-379e1e54b0ec None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:36.611483 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e249e3-796a-4433-b502-379e1e54b0ec None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:36.611918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e249e3-796a-4433-b502-379e1e54b0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:36.653228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f5e249e3-796a-4433-b502-379e1e54b0ec None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:36.656618 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f5e249e3-796a-4433-b502-379e1e54b0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:36.846977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 845/1680] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:36.855941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867b949-5c0f-4f2a-be7d-07218982b6aa None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:36.856300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867b949-5c0f-4f2a-be7d-07218982b6aa None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:36.856624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867b949-5c0f-4f2a-be7d-07218982b6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:36.899998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b867b949-5c0f-4f2a-be7d-07218982b6aa None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:36.903794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b867b949-5c0f-4f2a-be7d-07218982b6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:36.924209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 836/1681] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:36.929211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e249e3-796a-4433-b502-379e1e54b0ec None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:36.941465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e249e3-796a-4433-b502-379e1e54b0ec None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:36.959117 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b718fc-a63b-470a-9259-f2303e6cf92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:36.961531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b718fc-a63b-470a-9259-f2303e6cf92b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:36.968308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b718fc-a63b-470a-9259-f2303e6cf92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:36.969138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b718fc-a63b-470a-9259-f2303e6cf92b admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:36.970967 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b718fc-a63b-470a-9259-f2303e6cf92b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:08:36.973821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b718fc-a63b-470a-9259-f2303e6cf92b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:36.976797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b718fc-a63b-470a-9259-f2303e6cf92b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:37.008854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 846/1682] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:08:37.014184 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867b949-5c0f-4f2a-be7d-07218982b6aa None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:37.031389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867b949-5c0f-4f2a-be7d-07218982b6aa None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:37.047136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c304d12-8d06-46ea-b7b3-2fd792c1abc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:37.048074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c304d12-8d06-46ea-b7b3-2fd792c1abc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:37.048420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c304d12-8d06-46ea-b7b3-2fd792c1abc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:37.048927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c304d12-8d06-46ea-b7b3-2fd792c1abc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:37.051090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c304d12-8d06-46ea-b7b3-2fd792c1abc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:37.055382 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c304d12-8d06-46ea-b7b3-2fd792c1abc9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:37.127499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 837/1683] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:37 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:37.129907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b718fc-a63b-470a-9259-f2303e6cf92b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:37.152807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b718fc-a63b-470a-9259-f2303e6cf92b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:37.339437 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e04cbd-8a38-490b-be11-aafbb7d21a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:37.340247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e04cbd-8a38-490b-be11-aafbb7d21a36 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:37.340604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e04cbd-8a38-490b-be11-aafbb7d21a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:37.340952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e04cbd-8a38-490b-be11-aafbb7d21a36 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:37.343256 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e04cbd-8a38-490b-be11-aafbb7d21a36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:37.356054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e04cbd-8a38-490b-be11-aafbb7d21a36 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:37.406913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 847/1684] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:37 2022] POST /identity/v3/users => generated 291 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:08:37.408274 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c304d12-8d06-46ea-b7b3-2fd792c1abc9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:37.422774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c304d12-8d06-46ea-b7b3-2fd792c1abc9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:37.437890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b27eb-d23c-4eef-a194-6ca0c1c085c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:37.439142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b27eb-d23c-4eef-a194-6ca0c1c085c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:37.439638 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b27eb-d23c-4eef-a194-6ca0c1c085c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:37.440240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b27eb-d23c-4eef-a194-6ca0c1c085c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:37.441359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b27eb-d23c-4eef-a194-6ca0c1c085c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:37.444431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b27eb-d23c-4eef-a194-6ca0c1c085c8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:37.475199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 838/1685] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:37.477326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e04cbd-8a38-490b-be11-aafbb7d21a36 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:37.509974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e04cbd-8a38-490b-be11-aafbb7d21a36 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:37.530687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9b4f8-11f5-43c7-820b-0c550ce27499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:37.531531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9b4f8-11f5-43c7-820b-0c550ce27499 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:37.531887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9b4f8-11f5-43c7-820b-0c550ce27499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:37.532270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9b4f8-11f5-43c7-820b-0c550ce27499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:37.533859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9b4f8-11f5-43c7-820b-0c550ce27499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:37.536747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9b4f8-11f5-43c7-820b-0c550ce27499 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:37.558594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 848/1686] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:37.563028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b27eb-d23c-4eef-a194-6ca0c1c085c8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:37.574651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b27eb-d23c-4eef-a194-6ca0c1c085c8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:37.579957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52835c8-fbac-4093-aed6-3133cee36a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:37.581345 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52835c8-fbac-4093-aed6-3133cee36a24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:37.581814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52835c8-fbac-4093-aed6-3133cee36a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:37.582168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52835c8-fbac-4093-aed6-3133cee36a24 admin admin] PATH_INFO: `/v3/projects/d527c6b6d2a04c928f09d84af9449434/users/6c4e0c9a63074b3b87a0aff65e243589/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:37.612818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52835c8-fbac-4093-aed6-3133cee36a24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d527c6b6d2a04c928f09d84af9449434, user_id=6c4e0c9a63074b3b87a0aff65e243589, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:37.616149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52835c8-fbac-4093-aed6-3133cee36a24 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:37.637960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 839/1687] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:37 2022] PUT /identity/v3/projects/d527c6b6d2a04c928f09d84af9449434/users/6c4e0c9a63074b3b87a0aff65e243589/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:37.650572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e1dfd-316d-4fa9-940a-c580b894c3fe None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:37.651131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e1dfd-316d-4fa9-940a-c580b894c3fe None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:37.651131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e1dfd-316d-4fa9-940a-c580b894c3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:37.703874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:37.725168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-570e1dfd-316d-4fa9-940a-c580b894c3fe None None] MFA Rules not processed for user `6c4e0c9a63074b3b87a0aff65e243589`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:37.733499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-570e1dfd-316d-4fa9-940a-c580b894c3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:37.900628 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 849/1688] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:37.905444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52835c8-fbac-4093-aed6-3133cee36a24 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:38.001751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:38.021030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e1dfd-316d-4fa9-940a-c580b894c3fe None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:38.022526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-570e1dfd-316d-4fa9-940a-c580b894c3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:38.086303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52835c8-fbac-4093-aed6-3133cee36a24 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:38.096291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a759ed1-0712-41a1-8928-d978c61b34b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:38.102377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a759ed1-0712-41a1-8928-d978c61b34b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:38.102667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a759ed1-0712-41a1-8928-d978c61b34b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:38.103457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a759ed1-0712-41a1-8928-d978c61b34b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:38.104193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a759ed1-0712-41a1-8928-d978c61b34b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:38.109633 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a759ed1-0712-41a1-8928-d978c61b34b9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:38.172451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 840/1689] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:37 2022] POST /identity/v3/projects => generated 381 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:38.180148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a759ed1-0712-41a1-8928-d978c61b34b9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:38.180925 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e1dfd-316d-4fa9-940a-c580b894c3fe None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:38.331135 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533aebb-5cf8-4119-a0e2-eee7a779f95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:38.333460 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533aebb-5cf8-4119-a0e2-eee7a779f95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:38.333823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533aebb-5cf8-4119-a0e2-eee7a779f95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:38.334168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533aebb-5cf8-4119-a0e2-eee7a779f95b admin admin] PATH_INFO: `/v3/projects/81e9459d637748ffb6161ea762d9e2c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:38.337662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8533aebb-5cf8-4119-a0e2-eee7a779f95b admin admin] RBAC: Authorizing `identity:delete_project(project_id=81e9459d637748ffb6161ea762d9e2c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:38.340400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8533aebb-5cf8-4119-a0e2-eee7a779f95b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:38.341936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a759ed1-0712-41a1-8928-d978c61b34b9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:38.361188 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ad20a-3914-4c74-81f7-a87ef180a72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:38.362034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ad20a-3914-4c74-81f7-a87ef180a72f admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:38.362258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ad20a-3914-4c74-81f7-a87ef180a72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:38.362533 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ad20a-3914-4c74-81f7-a87ef180a72f admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:38.363237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ad20a-3914-4c74-81f7-a87ef180a72f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:38.365210 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ad20a-3914-4c74-81f7-a87ef180a72f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:38.412679 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 841/1690] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:38 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:38.422386 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b590e-898a-49c6-a3de-25b852a719d7 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:38.422869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b590e-898a-49c6-a3de-25b852a719d7 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:38.424282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b590e-898a-49c6-a3de-25b852a719d7 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:38.430652 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 842/1691] 10.222.0.79 () {54 vars in 921 bytes} [Mon Jul 4 10:08:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:08:38.433449 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b590e-898a-49c6-a3de-25b852a719d7 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:38.440497 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:38.443957 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8533aebb-5cf8-4119-a0e2-eee7a779f95b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81e9459d637748ffb6161ea762d9e2c3', 'request_id': 'req-8533aebb-5cf8-4119-a0e2-eee7a779f95b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:38.456914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8533aebb-5cf8-4119-a0e2-eee7a779f95b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81e9459d637748ffb6161ea762d9e2c3', 'request_id': 'req-8533aebb-5cf8-4119-a0e2-eee7a779f95b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:38.473839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8533aebb-5cf8-4119-a0e2-eee7a779f95b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81e9459d637748ffb6161ea762d9e2c3', 'request_id': 'req-8533aebb-5cf8-4119-a0e2-eee7a779f95b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:38.500087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8533aebb-5cf8-4119-a0e2-eee7a779f95b admin admin] The token cache is being invalidate because project 81e9459d637748ffb6161ea762d9e2c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:08:38.500445 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8533aebb-5cf8-4119-a0e2-eee7a779f95b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:38.501109 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8533aebb-5cf8-4119-a0e2-eee7a779f95b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:38.509239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 850/1692] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:08:38 2022] DELETE /identity/v3/projects/81e9459d637748ffb6161ea762d9e2c3 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:08:38.509239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd19e2-6421-4737-8fdd-35d2a5e198f7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:38.511424 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd19e2-6421-4737-8fdd-35d2a5e198f7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:38.511733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd19e2-6421-4737-8fdd-35d2a5e198f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:38.561801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-20bd19e2-6421-4737-8fdd-35d2a5e198f7 None None] MFA Rules not processed for user `5dc48318ab7944c197bf5f7673511931`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:38.566407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-20bd19e2-6421-4737-8fdd-35d2a5e198f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:38.613949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b590e-898a-49c6-a3de-25b852a719d7 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:38.623430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf897c1-62dc-4a65-9b9a-703075ca9cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:38.624142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf897c1-62dc-4a65-9b9a-703075ca9cac admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:38.624504 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf897c1-62dc-4a65-9b9a-703075ca9cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:38.624747 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf897c1-62dc-4a65-9b9a-703075ca9cac admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:38.625495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf897c1-62dc-4a65-9b9a-703075ca9cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:38.628298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf897c1-62dc-4a65-9b9a-703075ca9cac admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:38.674934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 843/1693] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:38.676611 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf897c1-62dc-4a65-9b9a-703075ca9cac admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:38.678582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4bf897c1-62dc-4a65-9b9a-703075ca9cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:38.698022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf897c1-62dc-4a65-9b9a-703075ca9cac admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:38.711413 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:38.713186 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:38.714450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:38.715741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] PATH_INFO: `/v3/users/0864360df3504c18b05b04e3bb30655c` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:38.718653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0864360df3504c18b05b04e3bb30655c)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:38.721535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:38.792034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:38.807619 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 851/1694] 10.222.0.79 () {58 vars in 1018 bytes} [Mon Jul 4 10:08:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:38.809282 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd19e2-6421-4737-8fdd-35d2a5e198f7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:38.813672 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-20bd19e2-6421-4737-8fdd-35d2a5e198f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:38.820799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0864360df3504c18b05b04e3bb30655c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:38.822959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0864360df3504c18b05b04e3bb30655c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:38.850954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0864360df3504c18b05b04e3bb30655c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:38.870894 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd19e2-6421-4737-8fdd-35d2a5e198f7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:38.884616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:38.886675 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:38.886966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:38.887249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] PATH_INFO: `/v3/users/451a736eb35d4a6198208ad22e71abc6` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:38.889605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0864360df3504c18b05b04e3bb30655c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:38.890673 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0864360df3504c18b05b04e3bb30655c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:38.892604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=451a736eb35d4a6198208ad22e71abc6)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:38.906887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:38.955387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 844/1695] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:08:38 2022] DELETE /identity/v3/users/0864360df3504c18b05b04e3bb30655c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:38.956686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:38.962390 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:39.038859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '451a736eb35d4a6198208ad22e71abc6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:39.041888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '451a736eb35d4a6198208ad22e71abc6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:39.059453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '451a736eb35d4a6198208ad22e71abc6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:39.083280 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '451a736eb35d4a6198208ad22e71abc6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:39.084249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '451a736eb35d4a6198208ad22e71abc6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:39.103833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85be37-d55a-4475-86cc-f7a0cb9d2084 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:39.111811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cf987-768c-4e95-be52-7ea518a60413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:39.112836 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cf987-768c-4e95-be52-7ea518a60413 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:39.113146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cf987-768c-4e95-be52-7ea518a60413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:39.113527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cf987-768c-4e95-be52-7ea518a60413 admin admin] PATH_INFO: `/v3/projects/570f0577f31f42c0b890084f308a7c5b/users/d7368028d4be4d20af118809edab2ffe/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:39.116806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 852/1696] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:08:38 2022] DELETE /identity/v3/users/451a736eb35d4a6198208ad22e71abc6 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:39.117813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:39.119238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:39.152053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cf987-768c-4e95-be52-7ea518a60413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=570f0577f31f42c0b890084f308a7c5b, user_id=d7368028d4be4d20af118809edab2ffe, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:39.157236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cf987-768c-4e95-be52-7ea518a60413 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:39.181686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 845/1697] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:38 2022] PUT /identity/v3/projects/570f0577f31f42c0b890084f308a7c5b/users/d7368028d4be4d20af118809edab2ffe/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:39.186399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cf987-768c-4e95-be52-7ea518a60413 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:39.189670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5c8cf987-768c-4e95-be52-7ea518a60413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:39.218776 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:39.256709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1b8b0-0493-4bec-943d-9301577576b5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:39.379193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6b7e6-9dce-4863-920d-281e54b5bb2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:39.383270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cf987-768c-4e95-be52-7ea518a60413 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:39.385752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6b7e6-9dce-4863-920d-281e54b5bb2c service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:39.386277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6b7e6-9dce-4863-920d-281e54b5bb2c service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:39.386664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6b7e6-9dce-4863-920d-281e54b5bb2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:39.388905 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c8b6b7e6-9dce-4863-920d-281e54b5bb2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:39.391829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46c4ea-e44c-4af9-923f-6064a3450d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:39.392555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46c4ea-e44c-4af9-923f-6064a3450d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:39.392959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46c4ea-e44c-4af9-923f-6064a3450d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:39.393308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46c4ea-e44c-4af9-923f-6064a3450d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:39.394221 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46c4ea-e44c-4af9-923f-6064a3450d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:39.398422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46c4ea-e44c-4af9-923f-6064a3450d1f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:39.413777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6b7e6-9dce-4863-920d-281e54b5bb2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:39.420859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6b7e6-9dce-4863-920d-281e54b5bb2c service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:39.429877 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 846/1698] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:39.435357 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46c4ea-e44c-4af9-923f-6064a3450d1f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:39.437992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 853/1699] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:39.465212 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46c4ea-e44c-4af9-923f-6064a3450d1f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:39.506046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10639c3-0b20-4d05-974a-77b6afccd596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:39.509287 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10639c3-0b20-4d05-974a-77b6afccd596 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:39.510214 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10639c3-0b20-4d05-974a-77b6afccd596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:39.510543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10639c3-0b20-4d05-974a-77b6afccd596 admin admin] PATH_INFO: `/v3/projects/570f0577f31f42c0b890084f308a7c5b/users/d7368028d4be4d20af118809edab2ffe/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:39.526498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10639c3-0b20-4d05-974a-77b6afccd596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=570f0577f31f42c0b890084f308a7c5b, user_id=d7368028d4be4d20af118809edab2ffe, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:39.530913 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10639c3-0b20-4d05-974a-77b6afccd596 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:39.558932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 847/1700] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:39 2022] PUT /identity/v3/projects/570f0577f31f42c0b890084f308a7c5b/users/d7368028d4be4d20af118809edab2ffe/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:39.570342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a7a50-b222-4100-8ad3-f06f7a15d3fa None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:39.570640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a7a50-b222-4100-8ad3-f06f7a15d3fa None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:39.570924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a7a50-b222-4100-8ad3-f06f7a15d3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:39.659835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e78a7a50-b222-4100-8ad3-f06f7a15d3fa None None] MFA Rules not processed for user `d7368028d4be4d20af118809edab2ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:39.667465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e78a7a50-b222-4100-8ad3-f06f7a15d3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:39.856640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 854/1701] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:39.864120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10639c3-0b20-4d05-974a-77b6afccd596 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:39.866824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a10639c3-0b20-4d05-974a-77b6afccd596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:39.905865 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:39.965182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10639c3-0b20-4d05-974a-77b6afccd596 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:39.976425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bbad5-4903-4f8c-9d0b-6771e07ee48b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:39.976425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bbad5-4903-4f8c-9d0b-6771e07ee48b service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:39.976708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bbad5-4903-4f8c-9d0b-6771e07ee48b service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:39.978094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bbad5-4903-4f8c-9d0b-6771e07ee48b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:40.001634 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6bbad5-4903-4f8c-9d0b-6771e07ee48b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:40.006214 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6bbad5-4903-4f8c-9d0b-6771e07ee48b service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:40.033130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 848/1702] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:40.186974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a7a50-b222-4100-8ad3-f06f7a15d3fa None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:40.204393 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a7a50-b222-4100-8ad3-f06f7a15d3fa None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:40.217007 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ff293-0acb-42c2-a8e2-1f1e11266517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:40.218123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ff293-0acb-42c2-a8e2-1f1e11266517 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:40.218615 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ff293-0acb-42c2-a8e2-1f1e11266517 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:40.219093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ff293-0acb-42c2-a8e2-1f1e11266517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:40.223194 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-221ff293-0acb-42c2-a8e2-1f1e11266517 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:40.332153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ff293-0acb-42c2-a8e2-1f1e11266517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:40.334117 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ff293-0acb-42c2-a8e2-1f1e11266517 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:40.340479 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:40.349169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 855/1703] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:40.545890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c54a65-4ab0-45b9-8193-073bece40b3a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:40.546471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c54a65-4ab0-45b9-8193-073bece40b3a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:40.546471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c54a65-4ab0-45b9-8193-073bece40b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:40.618829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-57c54a65-4ab0-45b9-8193-073bece40b3a None None] MFA Rules not processed for user `fc9c26318d6c41f6b9455153f0d0e55b`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:40.620064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-57c54a65-4ab0-45b9-8193-073bece40b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:40.810821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 849/1704] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:40.812696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4a4ef-b654-4559-a244-2638f15ea6f2 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:40.813132 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4a4ef-b654-4559-a244-2638f15ea6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:40.813565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4a4ef-b654-4559-a244-2638f15ea6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:40.830512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c54a65-4ab0-45b9-8193-073bece40b3a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:40.838527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-57c54a65-4ab0-45b9-8193-073bece40b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:40.868563 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ddb4a4ef-b654-4559-a244-2638f15ea6f2 None None] MFA Rules not processed for user `fc9c26318d6c41f6b9455153f0d0e55b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:40.871457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ddb4a4ef-b654-4559-a244-2638f15ea6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:40.900297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 856/1705] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:40.913562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4a4ef-b654-4559-a244-2638f15ea6f2 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:40.917495 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ddb4a4ef-b654-4559-a244-2638f15ea6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:41.024323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c54a65-4ab0-45b9-8193-073bece40b3a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:41.031160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40b788-2c82-4fca-b4e0-60317ebad373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:41.032075 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40b788-2c82-4fca-b4e0-60317ebad373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:41.032358 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40b788-2c82-4fca-b4e0-60317ebad373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:41.032633 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40b788-2c82-4fca-b4e0-60317ebad373 admin admin] PATH_INFO: `/v3/projects/d7641a4e75fd407ab1718d3716b50b1e` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:41.034445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40b788-2c82-4fca-b4e0-60317ebad373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7641a4e75fd407ab1718d3716b50b1e)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:41.037037 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40b788-2c82-4fca-b4e0-60317ebad373 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:41.088917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4a4ef-b654-4559-a244-2638f15ea6f2 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:41.103847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32d88b-aa71-4f78-9906-2b0e8851f52e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:41.107440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32d88b-aa71-4f78-9906-2b0e8851f52e service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:41.110425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32d88b-aa71-4f78-9906-2b0e8851f52e service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:41.111004 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32d88b-aa71-4f78-9906-2b0e8851f52e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:41.122315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2a40b788-2c82-4fca-b4e0-60317ebad373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7641a4e75fd407ab1718d3716b50b1e', 'request_id': 'req-2a40b788-2c82-4fca-b4e0-60317ebad373'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:41.126479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:41.132551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32d88b-aa71-4f78-9906-2b0e8851f52e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:41.138022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2a40b788-2c82-4fca-b4e0-60317ebad373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7641a4e75fd407ab1718d3716b50b1e', 'request_id': 'req-2a40b788-2c82-4fca-b4e0-60317ebad373'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:41.138282 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32d88b-aa71-4f78-9906-2b0e8851f52e service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:41.162604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2a40b788-2c82-4fca-b4e0-60317ebad373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7641a4e75fd407ab1718d3716b50b1e', 'request_id': 'req-2a40b788-2c82-4fca-b4e0-60317ebad373'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:41.178545 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 857/1706] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:08:41.184410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582cfa7-b718-484f-a565-35599969de49 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:41.185049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582cfa7-b718-484f-a565-35599969de49 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:41.186050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582cfa7-b718-484f-a565-35599969de49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:41.191188 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2a40b788-2c82-4fca-b4e0-60317ebad373 admin admin] The token cache is being invalidate because project d7641a4e75fd407ab1718d3716b50b1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:08:41.191523 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2a40b788-2c82-4fca-b4e0-60317ebad373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:41.196610 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2a40b788-2c82-4fca-b4e0-60317ebad373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:41.207108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 850/1707] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:08:40 2022] DELETE /identity/v3/projects/d7641a4e75fd407ab1718d3716b50b1e => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:41.208941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176c20b-a4c4-400d-acb1-c763cf74b845 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:41.209254 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176c20b-a4c4-400d-acb1-c763cf74b845 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:41.210625 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176c20b-a4c4-400d-acb1-c763cf74b845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:41.271019 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c582cfa7-b718-484f-a565-35599969de49 None None] MFA Rules not processed for user `64835c93d17543668a8039caeeaa393c`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:41.274447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c582cfa7-b718-484f-a565-35599969de49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:41.277430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-0176c20b-a4c4-400d-acb1-c763cf74b845 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:41.280730 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0176c20b-a4c4-400d-acb1-c763cf74b845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:41.449583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 858/1708] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:41.459855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582cfa7-b718-484f-a565-35599969de49 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:41.459855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c582cfa7-b718-484f-a565-35599969de49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:41.501161 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 851/1709] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:41.503504 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f180a95-a860-476b-bfa6-3f774ff1e9ac None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:41.504013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f180a95-a860-476b-bfa6-3f774ff1e9ac None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:41.504516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f180a95-a860-476b-bfa6-3f774ff1e9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:41.536451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4f180a95-a860-476b-bfa6-3f774ff1e9ac None None] MFA Rules not processed for user `64835c93d17543668a8039caeeaa393c`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:41.538242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4f180a95-a860-476b-bfa6-3f774ff1e9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:41.566461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 852/1710] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:41.567347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f81b3-700a-4025-9c3f-6966f511ecb4 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:41.567631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f81b3-700a-4025-9c3f-6966f511ecb4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:41.567903 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f81b3-700a-4025-9c3f-6966f511ecb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:41.593777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582cfa7-b718-484f-a565-35599969de49 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:41.604231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:41.604700 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:41.604983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:41.605259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] PATH_INFO: `/v3/projects/448f0d3c51ad4785945698c31bf45ecb` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:41.606881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] RBAC: Authorizing `identity:delete_project(project_id=448f0d3c51ad4785945698c31bf45ecb)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:41.608857 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:41.614948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d59f81b3-700a-4025-9c3f-6966f511ecb4 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:41.616573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d59f81b3-700a-4025-9c3f-6966f511ecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:41.624956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 853/1711] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:41.625858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f81b3-700a-4025-9c3f-6966f511ecb4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:41.630801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d59f81b3-700a-4025-9c3f-6966f511ecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:41.644569 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:41.720733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '448f0d3c51ad4785945698c31bf45ecb', 'request_id': 'req-259319f9-0d1e-40cc-96ce-1e40722e1198'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:41.733395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '448f0d3c51ad4785945698c31bf45ecb', 'request_id': 'req-259319f9-0d1e-40cc-96ce-1e40722e1198'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:41.750887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '448f0d3c51ad4785945698c31bf45ecb', 'request_id': 'req-259319f9-0d1e-40cc-96ce-1e40722e1198'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:41.776310 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] The token cache is being invalidate because project 448f0d3c51ad4785945698c31bf45ecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:08:41.776637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:41.777349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:41.779258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 859/1712] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:08:41 2022] DELETE /identity/v3/projects/448f0d3c51ad4785945698c31bf45ecb => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:41.781164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:41.784204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:41.794992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f81b3-700a-4025-9c3f-6966f511ecb4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:41.916587 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:41.931402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc9d44-af81-4d47-badb-49e53d6011d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:41.932083 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc9d44-af81-4d47-badb-49e53d6011d5 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:41.932369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc9d44-af81-4d47-badb-49e53d6011d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:41.932649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc9d44-af81-4d47-badb-49e53d6011d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:41.935117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d0bc9d44-af81-4d47-badb-49e53d6011d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:41.954265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259319f9-0d1e-40cc-96ce-1e40722e1198 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:41.973440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290054e-69ea-4108-9fa5-ae595077683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:41.974507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290054e-69ea-4108-9fa5-ae595077683a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:41.974988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290054e-69ea-4108-9fa5-ae595077683a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:41.975469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290054e-69ea-4108-9fa5-ae595077683a admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:41.976577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290054e-69ea-4108-9fa5-ae595077683a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:08:41.977106 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290054e-69ea-4108-9fa5-ae595077683a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:41.978974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290054e-69ea-4108-9fa5-ae595077683a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:42.025289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 860/1713] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:08:42.030078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290054e-69ea-4108-9fa5-ae595077683a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:42.038223 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f290054e-69ea-4108-9fa5-ae595077683a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:42.067338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc9d44-af81-4d47-badb-49e53d6011d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:42.068996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc9d44-af81-4d47-badb-49e53d6011d5 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:42.086947 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 854/1714] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:42.087802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc9d44-af81-4d47-badb-49e53d6011d5 service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:42.094634 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290054e-69ea-4108-9fa5-ae595077683a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:42.107121 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:42.108996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc9d44-af81-4d47-badb-49e53d6011d5 service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:42.110332 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:42.111017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:42.111511 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] PATH_INFO: `/v3/users/8cc0b86d074b47cab29497264d3d935d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:42.115386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cc0b86d074b47cab29497264d3d935d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:42.121181 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:42.126174 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9e75c-3c3e-41e0-aaef-75d6a3f9014d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:42.127028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9e75c-3c3e-41e0-aaef-75d6a3f9014d admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:42.127312 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9e75c-3c3e-41e0-aaef-75d6a3f9014d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:42.127628 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9e75c-3c3e-41e0-aaef-75d6a3f9014d admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:42.128484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9e75c-3c3e-41e0-aaef-75d6a3f9014d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:42.137368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9e75c-3c3e-41e0-aaef-75d6a3f9014d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:42.178650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 855/1715] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:42 2022] POST /identity/v3/projects => generated 391 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:42.178650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c53381-484a-4778-9d47-213c43c0864d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:42.178650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c53381-484a-4778-9d47-213c43c0864d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:42.178650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c53381-484a-4778-9d47-213c43c0864d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:42.224533 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f2c53381-484a-4778-9d47-213c43c0864d None None] MFA Rules not processed for user `dfba262d439a4253bf1c2c0f114d0cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:42.226696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f2c53381-484a-4778-9d47-213c43c0864d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:42.262079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cc0b86d074b47cab29497264d3d935d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:42.263088 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cc0b86d074b47cab29497264d3d935d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:42.294737 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cc0b86d074b47cab29497264d3d935d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:42.320355 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cc0b86d074b47cab29497264d3d935d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:42.321375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cc0b86d074b47cab29497264d3d935d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:42.358151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 861/1716] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:08:42 2022] DELETE /identity/v3/users/8cc0b86d074b47cab29497264d3d935d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:42.365193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:42.371229 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:42.387954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:42.411428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 856/1717] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:42.511010 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926d9d1-bb2a-4c40-8186-424ba521e112 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:42.517951 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3f7d5-adeb-4c59-a085-6ee36e794d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:42.517951 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3f7d5-adeb-4c59-a085-6ee36e794d97 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:42.518696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3f7d5-adeb-4c59-a085-6ee36e794d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:42.518986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3f7d5-adeb-4c59-a085-6ee36e794d97 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:42.519710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f3f7d5-adeb-4c59-a085-6ee36e794d97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:42.522245 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f3f7d5-adeb-4c59-a085-6ee36e794d97 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:42.568189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 862/1718] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:42 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:42.571343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c53381-484a-4778-9d47-213c43c0864d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:42.671783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100f4db-f4fe-43ac-87ec-f891dd68544e None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:42.672747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100f4db-f4fe-43ac-87ec-f891dd68544e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:42.672747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100f4db-f4fe-43ac-87ec-f891dd68544e None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:42.677721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 863/1719] 10.222.0.79 () {54 vars in 912 bytes} [Mon Jul 4 10:08:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 10:08:42.684748 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4d17d-11c3-4739-b516-afd8715a7d36 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:42.686026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4d17d-11c3-4739-b516-afd8715a7d36 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:42.686507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4d17d-11c3-4739-b516-afd8715a7d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:42.744591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c53381-484a-4778-9d47-213c43c0864d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:42.757758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f96ca-9f0e-4dfc-8931-7cf8e0f0505d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:42.760792 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f96ca-9f0e-4dfc-8931-7cf8e0f0505d admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:42.761085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f96ca-9f0e-4dfc-8931-7cf8e0f0505d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:42.761437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f96ca-9f0e-4dfc-8931-7cf8e0f0505d admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:42.762166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f96ca-9f0e-4dfc-8931-7cf8e0f0505d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:42.763842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f96ca-9f0e-4dfc-8931-7cf8e0f0505d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:42.795650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-90e4d17d-11c3-4739-b516-afd8715a7d36 None None] MFA Rules not processed for user `3a1f11a4b8554a98b7626a82f01888c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:42.796885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-90e4d17d-11c3-4739-b516-afd8715a7d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:42.804060 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 857/1720] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:42.808324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f96ca-9f0e-4dfc-8931-7cf8e0f0505d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:42.839092 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f96ca-9f0e-4dfc-8931-7cf8e0f0505d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:42.861998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d76a31-d057-4802-9fcf-ff1f18461707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:42.863650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d76a31-d057-4802-9fcf-ff1f18461707 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:42.863939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d76a31-d057-4802-9fcf-ff1f18461707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:42.865791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d76a31-d057-4802-9fcf-ff1f18461707 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:42.869580 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d76a31-d057-4802-9fcf-ff1f18461707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:42.871677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d76a31-d057-4802-9fcf-ff1f18461707 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:42.875640 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:42.903938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 858/1721] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:42.904957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d76a31-d057-4802-9fcf-ff1f18461707 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:42.928095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d76a31-d057-4802-9fcf-ff1f18461707 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:42.938693 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0ec4d-8753-412d-9795-23e382bb563e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:42.939581 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0ec4d-8753-412d-9795-23e382bb563e admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:42.939863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0ec4d-8753-412d-9795-23e382bb563e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:42.940998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0ec4d-8753-412d-9795-23e382bb563e admin admin] PATH_INFO: `/v3/projects/9a9e3f23bd3f4d489010a69e7f0906e4/users/f51336aa2ca74094a000eb13016699bb/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:43.001800 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0ec4d-8753-412d-9795-23e382bb563e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a9e3f23bd3f4d489010a69e7f0906e4, user_id=f51336aa2ca74094a000eb13016699bb, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:43.011228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0ec4d-8753-412d-9795-23e382bb563e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:43.027191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 864/1722] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:08:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:43.055088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 859/1723] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:42 2022] PUT /identity/v3/projects/9a9e3f23bd3f4d489010a69e7f0906e4/users/f51336aa2ca74094a000eb13016699bb/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:43.071314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66194e4-96d0-4d29-8372-05a565e2ea53 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:43.071314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66194e4-96d0-4d29-8372-05a565e2ea53 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:43.071314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66194e4-96d0-4d29-8372-05a565e2ea53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:43.133455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f66194e4-96d0-4d29-8372-05a565e2ea53 None None] MFA Rules not processed for user `f51336aa2ca74094a000eb13016699bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:43.140401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f66194e4-96d0-4d29-8372-05a565e2ea53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:43.198826 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d769ff-00de-468c-90d8-63d93bd1e8f1 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:43.199294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d769ff-00de-468c-90d8-63d93bd1e8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:43.199692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d769ff-00de-468c-90d8-63d93bd1e8f1 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:43.202455 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 860/1724] 10.222.0.79 () {54 vars in 912 bytes} [Mon Jul 4 10:08:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:08:43.203479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d769ff-00de-468c-90d8-63d93bd1e8f1 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:43.206261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b0d769ff-00de-468c-90d8-63d93bd1e8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:43.401130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 865/1725] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:43.402588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b571f0-2ddd-4ffa-932f-34c1723c8f42 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:43.404742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b571f0-2ddd-4ffa-932f-34c1723c8f42 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:43.405045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b571f0-2ddd-4ffa-932f-34c1723c8f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:43.434234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d769ff-00de-468c-90d8-63d93bd1e8f1 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:43.446506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae437e-2e53-4208-8780-9cab1a5366ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:43.450203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae437e-2e53-4208-8780-9cab1a5366ee service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:43.450607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae437e-2e53-4208-8780-9cab1a5366ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:43.453961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae437e-2e53-4208-8780-9cab1a5366ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:43.454878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-39b571f0-2ddd-4ffa-932f-34c1723c8f42 None None] MFA Rules not processed for user `3a1f11a4b8554a98b7626a82f01888c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:43.456017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-39b571f0-2ddd-4ffa-932f-34c1723c8f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:43.468965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae437e-2e53-4208-8780-9cab1a5366ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:43.474010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae437e-2e53-4208-8780-9cab1a5366ee service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:43.479046 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 866/1726] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:08:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:43.487190 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b571f0-2ddd-4ffa-932f-34c1723c8f42 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:43.495056 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 861/1727] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:43.525145 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:43.620918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae437e-2e53-4208-8780-9cab1a5366ee service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:43.624216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b571f0-2ddd-4ffa-932f-34c1723c8f42 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:43.630686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45815ca-c58c-4733-9379-1fa1cb686d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:43.631703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45815ca-c58c-4733-9379-1fa1cb686d52 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:43.633013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae437e-2e53-4208-8780-9cab1a5366ee service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:43.633333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45815ca-c58c-4733-9379-1fa1cb686d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:43.633964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45815ca-c58c-4733-9379-1fa1cb686d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:43.635994 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45815ca-c58c-4733-9379-1fa1cb686d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:43.640964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45815ca-c58c-4733-9379-1fa1cb686d52 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:43.644421 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ef90f-c1a2-4aea-89ad-4da18203e3d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:43.645704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ef90f-c1a2-4aea-89ad-4da18203e3d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:43.645704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ef90f-c1a2-4aea-89ad-4da18203e3d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:43.645704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ef90f-c1a2-4aea-89ad-4da18203e3d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:43.665564 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ef90f-c1a2-4aea-89ad-4da18203e3d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:43.667711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ef90f-c1a2-4aea-89ad-4da18203e3d1 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:43.686761 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 867/1728] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:43 2022] POST /identity/v3/projects => generated 389 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:43.694091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45815ca-c58c-4733-9379-1fa1cb686d52 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:43.694991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 862/1729] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:43.811218 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45815ca-c58c-4733-9379-1fa1cb686d52 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:43.828248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66552a90-9d7f-4c86-ba68-567cf16088be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:43.828882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66552a90-9d7f-4c86-ba68-567cf16088be admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:43.829161 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66552a90-9d7f-4c86-ba68-567cf16088be admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:43.831710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66552a90-9d7f-4c86-ba68-567cf16088be admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:43.832764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66552a90-9d7f-4c86-ba68-567cf16088be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:43.836926 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66552a90-9d7f-4c86-ba68-567cf16088be admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:43.861043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ef90f-c1a2-4aea-89ad-4da18203e3d1 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:43.863511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e57ef90f-c1a2-4aea-89ad-4da18203e3d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:43.893247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ef90f-c1a2-4aea-89ad-4da18203e3d1 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:43.903459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 868/1730] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:43 2022] POST /identity/v3/users => generated 301 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:43.905459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66552a90-9d7f-4c86-ba68-567cf16088be admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:43.910437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed79901-6ca4-483b-9126-20a916db61b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:43.911567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed79901-6ca4-483b-9126-20a916db61b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:43.911994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed79901-6ca4-483b-9126-20a916db61b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:43.912402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed79901-6ca4-483b-9126-20a916db61b9 admin admin] PATH_INFO: `/v3/projects/711e448fb91f424b84aa2834e3a4ff37` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:43.914766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed79901-6ca4-483b-9126-20a916db61b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=711e448fb91f424b84aa2834e3a4ff37)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:43.916891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed79901-6ca4-483b-9126-20a916db61b9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:43.933845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66552a90-9d7f-4c86-ba68-567cf16088be admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:43.943885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5822e-eedd-48e8-9eb6-8d58e105787d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:43.944644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5822e-eedd-48e8-9eb6-8d58e105787d admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:43.944929 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5822e-eedd-48e8-9eb6-8d58e105787d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:43.945293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5822e-eedd-48e8-9eb6-8d58e105787d admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:43.946417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5822e-eedd-48e8-9eb6-8d58e105787d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:43.950426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5822e-eedd-48e8-9eb6-8d58e105787d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:44.012116 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 869/1731] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:08:44.013128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5822e-eedd-48e8-9eb6-8d58e105787d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:44.034387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5822e-eedd-48e8-9eb6-8d58e105787d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:44.050594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7ed79901-6ca4-483b-9126-20a916db61b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '711e448fb91f424b84aa2834e3a4ff37', 'request_id': 'req-7ed79901-6ca4-483b-9126-20a916db61b9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:44.057530 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7ed79901-6ca4-483b-9126-20a916db61b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '711e448fb91f424b84aa2834e3a4ff37', 'request_id': 'req-7ed79901-6ca4-483b-9126-20a916db61b9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:44.073298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7ed79901-6ca4-483b-9126-20a916db61b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '711e448fb91f424b84aa2834e3a4ff37', 'request_id': 'req-7ed79901-6ca4-483b-9126-20a916db61b9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:44.105136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7ed79901-6ca4-483b-9126-20a916db61b9 admin admin] The token cache is being invalidate because project 711e448fb91f424b84aa2834e3a4ff37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:08:44.105467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7ed79901-6ca4-483b-9126-20a916db61b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:44.106198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7ed79901-6ca4-483b-9126-20a916db61b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:44.113302 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 863/1732] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:08:43 2022] DELETE /identity/v3/projects/711e448fb91f424b84aa2834e3a4ff37 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:44.115211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ca074-f824-41da-88a5-072100f83547 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:44.115552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ca074-f824-41da-88a5-072100f83547 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:44.115836 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ca074-f824-41da-88a5-072100f83547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:44.160360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f71ca074-f824-41da-88a5-072100f83547 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:44.162481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f71ca074-f824-41da-88a5-072100f83547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:44.252799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bd526-b8d7-4969-a6f3-30a74ff86836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:44.253934 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bd526-b8d7-4969-a6f3-30a74ff86836 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:44.254308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bd526-b8d7-4969-a6f3-30a74ff86836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:44.254531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bd526-b8d7-4969-a6f3-30a74ff86836 admin admin] PATH_INFO: `/v3/projects/bba58db694bb40458fdf4913147aa16a/users/e27cec66d1074697aeecebd635793923/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:44.279228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:44.292840 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bd526-b8d7-4969-a6f3-30a74ff86836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba58db694bb40458fdf4913147aa16a, user_id=e27cec66d1074697aeecebd635793923, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:44.299248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bd526-b8d7-4969-a6f3-30a74ff86836 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:44.325182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:44.339410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 864/1733] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:44.345751 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 870/1734] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:44 2022] PUT /identity/v3/projects/bba58db694bb40458fdf4913147aa16a/users/e27cec66d1074697aeecebd635793923/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:44.346866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bd526-b8d7-4969-a6f3-30a74ff86836 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:44.347471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f491ae9-3330-4b2e-ba67-d3b0f19433c3 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:44.347717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f491ae9-3330-4b2e-ba67-d3b0f19433c3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:44.348000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f491ae9-3330-4b2e-ba67-d3b0f19433c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:44.351051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ff9bd526-b8d7-4969-a6f3-30a74ff86836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:44.422866 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5f491ae9-3330-4b2e-ba67-d3b0f19433c3 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:44.425729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5f491ae9-3330-4b2e-ba67-d3b0f19433c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:44.630434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bd526-b8d7-4969-a6f3-30a74ff86836 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:44.646337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655cafc-5217-4211-8092-8a511c41db1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:44.650493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 865/1735] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:44.653588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655cafc-5217-4211-8092-8a511c41db1c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:44.653988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655cafc-5217-4211-8092-8a511c41db1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:44.654279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655cafc-5217-4211-8092-8a511c41db1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:44.655075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655cafc-5217-4211-8092-8a511c41db1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:44.655670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f491ae9-3330-4b2e-ba67-d3b0f19433c3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:44.680416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f491ae9-3330-4b2e-ba67-d3b0f19433c3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:44.681553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655cafc-5217-4211-8092-8a511c41db1c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:44.707203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fef1de-13cf-45f6-a66b-3d7183982efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:44.707913 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fef1de-13cf-45f6-a66b-3d7183982efe admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:44.708198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fef1de-13cf-45f6-a66b-3d7183982efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:44.708476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fef1de-13cf-45f6-a66b-3d7183982efe admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:44.709227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fef1de-13cf-45f6-a66b-3d7183982efe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:08:44.709530 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fef1de-13cf-45f6-a66b-3d7183982efe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:44.711368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fef1de-13cf-45f6-a66b-3d7183982efe admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:44.730009 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 871/1736] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:44.734099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655cafc-5217-4211-8092-8a511c41db1c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:44.739125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 866/1737] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:08:44.743416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fef1de-13cf-45f6-a66b-3d7183982efe admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:44.770367 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655cafc-5217-4211-8092-8a511c41db1c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:44.775907 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fef1de-13cf-45f6-a66b-3d7183982efe admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:44.785707 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2388128-2b7c-4993-94fd-525ff860453a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:44.786688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2388128-2b7c-4993-94fd-525ff860453a admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:44.786920 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2388128-2b7c-4993-94fd-525ff860453a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:44.787198 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2388128-2b7c-4993-94fd-525ff860453a admin admin] PATH_INFO: `/v3/projects/bba58db694bb40458fdf4913147aa16a/users/e27cec66d1074697aeecebd635793923/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:44.790434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2388128-2b7c-4993-94fd-525ff860453a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba58db694bb40458fdf4913147aa16a, user_id=e27cec66d1074697aeecebd635793923, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:44.799632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2388128-2b7c-4993-94fd-525ff860453a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:44.806206 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492da7c-2de2-4117-82df-716752f1f857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:44.812513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492da7c-2de2-4117-82df-716752f1f857 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:44.818721 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492da7c-2de2-4117-82df-716752f1f857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:44.819011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492da7c-2de2-4117-82df-716752f1f857 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:44.822768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492da7c-2de2-4117-82df-716752f1f857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:44.824334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 872/1738] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:44 2022] PUT /identity/v3/projects/bba58db694bb40458fdf4913147aa16a/users/e27cec66d1074697aeecebd635793923/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:44.824773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492da7c-2de2-4117-82df-716752f1f857 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:44.835732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10bad7-87da-42be-8559-67b49748afb6 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:44.836047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10bad7-87da-42be-8559-67b49748afb6 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:44.836301 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10bad7-87da-42be-8559-67b49748afb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:44.900873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 867/1739] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:44 2022] POST /identity/v3/projects => generated 367 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:44.902256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf74a4-d023-4f42-82c4-4c822605f320 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:44.902548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf74a4-d023-4f42-82c4-4c822605f320 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:44.902829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf74a4-d023-4f42-82c4-4c822605f320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:44.909672 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-df10bad7-87da-42be-8559-67b49748afb6 None None] MFA Rules not processed for user `e27cec66d1074697aeecebd635793923`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:44.910924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-df10bad7-87da-42be-8559-67b49748afb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:44.965546 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-31bf74a4-d023-4f42-82c4-4c822605f320 None None] MFA Rules not processed for user `1c05b9b08df0437f9b0abb167d17aa56`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:44.967068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-31bf74a4-d023-4f42-82c4-4c822605f320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:45.063573 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 873/1740] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:45.064564 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10bad7-87da-42be-8559-67b49748afb6 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:45.143723 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 868/1741] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:45.144987 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf74a4-d023-4f42-82c4-4c822605f320 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:45.152367 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-31bf74a4-d023-4f42-82c4-4c822605f320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:45.243801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10bad7-87da-42be-8559-67b49748afb6 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:45.251051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d2ce5-fee3-4cf8-933e-3e03d826ea1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:45.251838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d2ce5-fee3-4cf8-933e-3e03d826ea1b admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:45.252187 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d2ce5-fee3-4cf8-933e-3e03d826ea1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:45.252658 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d2ce5-fee3-4cf8-933e-3e03d826ea1b admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:45.253887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d2ce5-fee3-4cf8-933e-3e03d826ea1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:45.259599 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d2ce5-fee3-4cf8-933e-3e03d826ea1b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:45.293496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:45.295520 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:45.307817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 874/1742] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:45 2022] POST /identity/v3/users => generated 284 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:45.309888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d2ce5-fee3-4cf8-933e-3e03d826ea1b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:45.327254 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf74a4-d023-4f42-82c4-4c822605f320 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:45.340502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b51fd-8668-4ab5-9d57-8891187f3861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:45.341195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b51fd-8668-4ab5-9d57-8891187f3861 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:45.341477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b51fd-8668-4ab5-9d57-8891187f3861 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:45.341874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b51fd-8668-4ab5-9d57-8891187f3861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:45.347104 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d2ce5-fee3-4cf8-933e-3e03d826ea1b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:45.364938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b51fd-8668-4ab5-9d57-8891187f3861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:45.366485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b51fd-8668-4ab5-9d57-8891187f3861 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:45.367998 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79d55b-0f03-4091-b54e-41b1b1c3dd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:45.368704 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79d55b-0f03-4091-b54e-41b1b1c3dd43 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:45.368994 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79d55b-0f03-4091-b54e-41b1b1c3dd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:45.370060 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79d55b-0f03-4091-b54e-41b1b1c3dd43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:45.377337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79d55b-0f03-4091-b54e-41b1b1c3dd43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:45.381473 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79d55b-0f03-4091-b54e-41b1b1c3dd43 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:45.391971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 869/1743] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:45.409592 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b51fd-8668-4ab5-9d57-8891187f3861 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:45.410583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 875/1744] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:45.435212 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b51fd-8668-4ab5-9d57-8891187f3861 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:45.443007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8d2a1-e560-43ca-b552-08455f52a760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:45.443655 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8d2a1-e560-43ca-b552-08455f52a760 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:45.443953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8d2a1-e560-43ca-b552-08455f52a760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:45.444229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8d2a1-e560-43ca-b552-08455f52a760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:45.444958 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8d2a1-e560-43ca-b552-08455f52a760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:45.446677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8d2a1-e560-43ca-b552-08455f52a760 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:45.466219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79d55b-0f03-4091-b54e-41b1b1c3dd43 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:45.479784 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 870/1745] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:45.483768 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79d55b-0f03-4091-b54e-41b1b1c3dd43 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:45.492678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8d2a1-e560-43ca-b552-08455f52a760 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:45.494394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-58c8d2a1-e560-43ca-b552-08455f52a760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:45.499307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eefc308-20bb-4b96-b966-391d69549dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:45.503699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefc308-20bb-4b96-b966-391d69549dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:45.503986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefc308-20bb-4b96-b966-391d69549dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:45.504264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefc308-20bb-4b96-b966-391d69549dbb admin admin] PATH_INFO: `/v3/projects/0ba469fa844c4bb0b80d3605ef36eec6/users/93a7eefdc67c4b798b491a6daff4b813/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:45.521582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8d2a1-e560-43ca-b552-08455f52a760 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:45.529450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd67dac-5d2c-4f85-b99d-a4b5c3457aa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:45.530182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd67dac-5d2c-4f85-b99d-a4b5c3457aa1 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:45.530615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd67dac-5d2c-4f85-b99d-a4b5c3457aa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:45.531524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd67dac-5d2c-4f85-b99d-a4b5c3457aa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:45.535325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2cd67dac-5d2c-4f85-b99d-a4b5c3457aa1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:45.541353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eefc308-20bb-4b96-b966-391d69549dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba469fa844c4bb0b80d3605ef36eec6, user_id=93a7eefdc67c4b798b491a6daff4b813, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:45.544792 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eefc308-20bb-4b96-b966-391d69549dbb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:45.580218 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 876/1746] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:45 2022] PUT /identity/v3/projects/0ba469fa844c4bb0b80d3605ef36eec6/users/93a7eefdc67c4b798b491a6daff4b813/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:45.594156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6140be1-1648-4861-af16-38f073d1b852 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:45.594447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6140be1-1648-4861-af16-38f073d1b852 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:45.594727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6140be1-1648-4861-af16-38f073d1b852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:45.635136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f6140be1-1648-4861-af16-38f073d1b852 None None] MFA Rules not processed for user `93a7eefdc67c4b798b491a6daff4b813`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:45.636547 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f6140be1-1648-4861-af16-38f073d1b852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:45.704438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd67dac-5d2c-4f85-b99d-a4b5c3457aa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:45.705972 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd67dac-5d2c-4f85-b99d-a4b5c3457aa1 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:45.743006 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 871/1747] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:45.781190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd67dac-5d2c-4f85-b99d-a4b5c3457aa1 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:45.781761 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 877/1748] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:45.797130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd67dac-5d2c-4f85-b99d-a4b5c3457aa1 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:45.940944 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dbce6-7585-4baf-ad61-e674ec32d789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:45.941705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dbce6-7585-4baf-ad61-e674ec32d789 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:45.941998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dbce6-7585-4baf-ad61-e674ec32d789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:45.947429 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dbce6-7585-4baf-ad61-e674ec32d789 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:45.948277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689dbce6-7585-4baf-ad61-e674ec32d789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:45.962736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689dbce6-7585-4baf-ad61-e674ec32d789 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:46.017020 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6140be1-1648-4861-af16-38f073d1b852 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:46.020284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f6140be1-1648-4861-af16-38f073d1b852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:46.074186 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 872/1749] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:08:45 2022] POST /identity/v3/projects => generated 367 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:46.075517 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dbce6-7585-4baf-ad61-e674ec32d789 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:46.078917 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-689dbce6-7585-4baf-ad61-e674ec32d789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:46.295234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6140be1-1648-4861-af16-38f073d1b852 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:46.316815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:46.375774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dbce6-7585-4baf-ad61-e674ec32d789 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:46.387344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bd28c-5358-4a34-8058-e0b6ee258ec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:46.388251 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bd28c-5358-4a34-8058-e0b6ee258ec3 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:46.389829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bd28c-5358-4a34-8058-e0b6ee258ec3 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:46.390312 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bd28c-5358-4a34-8058-e0b6ee258ec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:46.449856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:46.487247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c011ae-08eb-4996-9e14-1d6966bb2cf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:46.488036 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c011ae-08eb-4996-9e14-1d6966bb2cf6 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:46.488423 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c011ae-08eb-4996-9e14-1d6966bb2cf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:46.489022 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c011ae-08eb-4996-9e14-1d6966bb2cf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:46.494722 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-16c011ae-08eb-4996-9e14-1d6966bb2cf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:46.559273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068bd28c-5358-4a34-8058-e0b6ee258ec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:46.561156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068bd28c-5358-4a34-8058-e0b6ee258ec3 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:46.595394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 873/1750] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:46.598540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bd28c-5358-4a34-8058-e0b6ee258ec3 service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:46.697276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c011ae-08eb-4996-9e14-1d6966bb2cf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:46.701312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c011ae-08eb-4996-9e14-1d6966bb2cf6 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:46.738335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 878/1751] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:08:46.739721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c011ae-08eb-4996-9e14-1d6966bb2cf6 service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:46.752811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-16c011ae-08eb-4996-9e14-1d6966bb2cf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:46.809494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bd28c-5358-4a34-8058-e0b6ee258ec3 service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:46.832896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125511e-1bac-4aff-ae0a-be711fffc579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:46.835569 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125511e-1bac-4aff-ae0a-be711fffc579 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:46.835867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125511e-1bac-4aff-ae0a-be711fffc579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:46.836163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125511e-1bac-4aff-ae0a-be711fffc579 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:46.836885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125511e-1bac-4aff-ae0a-be711fffc579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:46.838927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125511e-1bac-4aff-ae0a-be711fffc579 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:46.908062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 874/1752] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:08:46 2022] POST /identity/v3/users => generated 290 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:08:46.910110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125511e-1bac-4aff-ae0a-be711fffc579 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:46.914359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8125511e-1bac-4aff-ae0a-be711fffc579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:47.008340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c011ae-08eb-4996-9e14-1d6966bb2cf6 service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:47.038732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed27b4-9003-4a7a-98fa-c8102bd1a016 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:47.040952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed27b4-9003-4a7a-98fa-c8102bd1a016 service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:47.042609 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed27b4-9003-4a7a-98fa-c8102bd1a016 service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:47.043754 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed27b4-9003-4a7a-98fa-c8102bd1a016 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:47.052123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c6ed27b4-9003-4a7a-98fa-c8102bd1a016 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:47.092401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed27b4-9003-4a7a-98fa-c8102bd1a016 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:47.096212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed27b4-9003-4a7a-98fa-c8102bd1a016 service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:47.134112 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 879/1753] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:47.140511 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faca0bf-9632-43b2-b1fe-1e62a70868b8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:47.141112 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faca0bf-9632-43b2-b1fe-1e62a70868b8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:47.142001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faca0bf-9632-43b2-b1fe-1e62a70868b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:47.187120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125511e-1bac-4aff-ae0a-be711fffc579 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:47.206823 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcb968-7741-44ba-b6ce-134d64f45828 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:47.207366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcb968-7741-44ba-b6ce-134d64f45828 service placement] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:47.207647 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcb968-7741-44ba-b6ce-134d64f45828 service placement] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:47.207922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcb968-7741-44ba-b6ce-134d64f45828 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:47.235439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7faca0bf-9632-43b2-b1fe-1e62a70868b8 None None] MFA Rules not processed for user `88d2351fc3b74ef5a530fded1be6b997`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:47.242117 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7faca0bf-9632-43b2-b1fe-1e62a70868b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:47.245129 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bcb968-7741-44ba-b6ce-134d64f45828 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:47.247776 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bcb968-7741-44ba-b6ce-134d64f45828 service placement] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:47.299925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 875/1754] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:47.302759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcb968-7741-44ba-b6ce-134d64f45828 service placement] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:47.342876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcb968-7741-44ba-b6ce-134d64f45828 service placement] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:47.375373 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78ee41-7c0c-47dc-a27f-3ec9be3edd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:47.376079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78ee41-7c0c-47dc-a27f-3ec9be3edd23 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:47.376373 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78ee41-7c0c-47dc-a27f-3ec9be3edd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:47.376646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78ee41-7c0c-47dc-a27f-3ec9be3edd23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:47.384570 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78ee41-7c0c-47dc-a27f-3ec9be3edd23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:47.394477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78ee41-7c0c-47dc-a27f-3ec9be3edd23 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:47.423083 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:47.435842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 880/1755] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:47.438125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756884e7-a65f-4e3e-9546-0316d31f5819 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:47.438555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756884e7-a65f-4e3e-9546-0316d31f5819 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:47.439003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756884e7-a65f-4e3e-9546-0316d31f5819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:47.452130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:47.452725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 876/1756] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:08:47.455193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78ee41-7c0c-47dc-a27f-3ec9be3edd23 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:47.495284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78ee41-7c0c-47dc-a27f-3ec9be3edd23 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:47.515345 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688db8b1-3a54-4912-9207-52ef318f8657 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:47.516250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688db8b1-3a54-4912-9207-52ef318f8657 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:47.516634 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688db8b1-3a54-4912-9207-52ef318f8657 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:47.516873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688db8b1-3a54-4912-9207-52ef318f8657 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:47.543369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688db8b1-3a54-4912-9207-52ef318f8657 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:47.548136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688db8b1-3a54-4912-9207-52ef318f8657 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:47.575636 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 877/1757] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:47.577567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688db8b1-3a54-4912-9207-52ef318f8657 service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:47.620342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688db8b1-3a54-4912-9207-52ef318f8657 service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:47.627461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cdf79-35f7-463d-b927-134e6ebbb51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:47.628449 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cdf79-35f7-463d-b927-134e6ebbb51e admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:47.628929 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cdf79-35f7-463d-b927-134e6ebbb51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:47.629206 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cdf79-35f7-463d-b927-134e6ebbb51e admin admin] PATH_INFO: `/v3/projects/e5ad7ae2e74e40e89ba11f30d7089a2b/users/278805d10f744e2f97d656fa6ef0abfe/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:47.679715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cdf79-35f7-463d-b927-134e6ebbb51e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ad7ae2e74e40e89ba11f30d7089a2b, user_id=278805d10f744e2f97d656fa6ef0abfe, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:47.696626 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cdf79-35f7-463d-b927-134e6ebbb51e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:47.757665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 878/1758] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:47 2022] PUT /identity/v3/projects/e5ad7ae2e74e40e89ba11f30d7089a2b/users/278805d10f744e2f97d656fa6ef0abfe/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:47.759684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cdf79-35f7-463d-b927-134e6ebbb51e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:47.842084 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-756884e7-a65f-4e3e-9546-0316d31f5819 None None] MFA Rules not processed for user `fc9c26318d6c41f6b9455153f0d0e55b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:47.847103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-756884e7-a65f-4e3e-9546-0316d31f5819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:47.978048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cdf79-35f7-463d-b927-134e6ebbb51e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:47.996046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2125b-f558-4212-ae0b-34f0253a5ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:47.996873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2125b-f558-4212-ae0b-34f0253a5ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:47.997263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2125b-f558-4212-ae0b-34f0253a5ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:47.997732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2125b-f558-4212-ae0b-34f0253a5ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:47.998583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f2125b-f558-4212-ae0b-34f0253a5ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:48.000333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f2125b-f558-4212-ae0b-34f0253a5ba8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:48.012479 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 881/1759] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 575 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:48.027536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756884e7-a65f-4e3e-9546-0316d31f5819 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:48.027999 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 879/1760] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:08:48.041393 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756884e7-a65f-4e3e-9546-0316d31f5819 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:48.058840 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba60db2-72bb-47ca-837a-e8d4de8fbc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:48.060018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60db2-72bb-47ca-837a-e8d4de8fbc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:48.061412 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60db2-72bb-47ca-837a-e8d4de8fbc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:48.061945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60db2-72bb-47ca-837a-e8d4de8fbc70 admin admin] PATH_INFO: `/v3/projects/e5ad7ae2e74e40e89ba11f30d7089a2b/users/278805d10f744e2f97d656fa6ef0abfe/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:48.066593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba60db2-72bb-47ca-837a-e8d4de8fbc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ad7ae2e74e40e89ba11f30d7089a2b, user_id=278805d10f744e2f97d656fa6ef0abfe, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:48.073436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba60db2-72bb-47ca-837a-e8d4de8fbc70 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:48.105934 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 882/1761] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:08:48 2022] PUT /identity/v3/projects/e5ad7ae2e74e40e89ba11f30d7089a2b/users/278805d10f744e2f97d656fa6ef0abfe/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:48.116828 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c149b-175c-40f8-b460-dbf441e05f54 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:48.117279 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c149b-175c-40f8-b460-dbf441e05f54 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:48.117769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c149b-175c-40f8-b460-dbf441e05f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:48.187383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c42c149b-175c-40f8-b460-dbf441e05f54 None None] MFA Rules not processed for user `278805d10f744e2f97d656fa6ef0abfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:48.193738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c42c149b-175c-40f8-b460-dbf441e05f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:48.390176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 880/1762] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:48 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:48.419451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba60db2-72bb-47ca-837a-e8d4de8fbc70 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:48.654823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba60db2-72bb-47ca-837a-e8d4de8fbc70 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:48.664274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e4ba8-c4fa-4962-9654-c2b6666e320c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:48.664983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e4ba8-c4fa-4962-9654-c2b6666e320c service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:48.665278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e4ba8-c4fa-4962-9654-c2b6666e320c service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:48.668844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e4ba8-c4fa-4962-9654-c2b6666e320c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:48.731896 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e4ba8-c4fa-4962-9654-c2b6666e320c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:48.739842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e4ba8-c4fa-4962-9654-c2b6666e320c service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:48.788429 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 883/1763] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:48 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:08:48.900890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c149b-175c-40f8-b460-dbf441e05f54 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:48.918847 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c149b-175c-40f8-b460-dbf441e05f54 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:48.930802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783607c-0bb3-4cc1-a2cd-35ad60463097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:48.933033 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783607c-0bb3-4cc1-a2cd-35ad60463097 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:48.933326 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783607c-0bb3-4cc1-a2cd-35ad60463097 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:48.936135 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783607c-0bb3-4cc1-a2cd-35ad60463097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:49.143247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:49.174515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783607c-0bb3-4cc1-a2cd-35ad60463097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:49.176287 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783607c-0bb3-4cc1-a2cd-35ad60463097 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:49.188555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 881/1764] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:08:51.954759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c003c-ed50-4b8f-9e33-636c30f397d0 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:51.955390 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c003c-ed50-4b8f-9e33-636c30f397d0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:51.955716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c003c-ed50-4b8f-9e33-636c30f397d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:52.009524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-890c003c-ed50-4b8f-9e33-636c30f397d0 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:52.010812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-890c003c-ed50-4b8f-9e33-636c30f397d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:52.212306 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 884/1765] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:08:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:52.550947 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783607c-0bb3-4cc1-a2cd-35ad60463097 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:52.553157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d783607c-0bb3-4cc1-a2cd-35ad60463097 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:52.837499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783607c-0bb3-4cc1-a2cd-35ad60463097 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:52.871898 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54097c4e-fca2-4422-a43d-a913c84559e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:52.872972 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54097c4e-fca2-4422-a43d-a913c84559e2 service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:52.873452 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54097c4e-fca2-4422-a43d-a913c84559e2 service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:52.873897 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54097c4e-fca2-4422-a43d-a913c84559e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:52.895714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54097c4e-fca2-4422-a43d-a913c84559e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:52.897388 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54097c4e-fca2-4422-a43d-a913c84559e2 service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:52.928433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 882/1766] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:08:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 10:08:53.502261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c003c-ed50-4b8f-9e33-636c30f397d0 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:53.504560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-890c003c-ed50-4b8f-9e33-636c30f397d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:53.625663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:53.696290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c003c-ed50-4b8f-9e33-636c30f397d0 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:53.717124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acae6ed-0bdf-4de6-a7f9-a1bc9a10c080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:53.719753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acae6ed-0bdf-4de6-a7f9-a1bc9a10c080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:53.721197 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acae6ed-0bdf-4de6-a7f9-a1bc9a10c080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:53.721973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acae6ed-0bdf-4de6-a7f9-a1bc9a10c080 admin admin] PATH_INFO: `/v3/users/6c4e0c9a63074b3b87a0aff65e243589` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:53.726338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acae6ed-0bdf-4de6-a7f9-a1bc9a10c080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c4e0c9a63074b3b87a0aff65e243589)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:53.740257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acae6ed-0bdf-4de6-a7f9-a1bc9a10c080 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:53.748265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214bd8a-909d-4a27-9dbc-1dd9ffe860b3 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:53.749580 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214bd8a-909d-4a27-9dbc-1dd9ffe860b3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:53.750317 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214bd8a-909d-4a27-9dbc-1dd9ffe860b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:53.802808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-8214bd8a-909d-4a27-9dbc-1dd9ffe860b3 None None] MFA Rules not processed for user `dfba262d439a4253bf1c2c0f114d0cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:53.809660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8214bd8a-909d-4a27-9dbc-1dd9ffe860b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:53.959128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8acae6ed-0bdf-4de6-a7f9-a1bc9a10c080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c4e0c9a63074b3b87a0aff65e243589'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:53.962388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8acae6ed-0bdf-4de6-a7f9-a1bc9a10c080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c4e0c9a63074b3b87a0aff65e243589'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:53.999222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8acae6ed-0bdf-4de6-a7f9-a1bc9a10c080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c4e0c9a63074b3b87a0aff65e243589'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:54.092808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8acae6ed-0bdf-4de6-a7f9-a1bc9a10c080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c4e0c9a63074b3b87a0aff65e243589'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:54.096259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8acae6ed-0bdf-4de6-a7f9-a1bc9a10c080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c4e0c9a63074b3b87a0aff65e243589'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:54.151104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 883/1767] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:54.152354 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214bd8a-909d-4a27-9dbc-1dd9ffe860b3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:54.179778 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214bd8a-909d-4a27-9dbc-1dd9ffe860b3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:54.198697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:54.199852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:54.200275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:54.200681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] PATH_INFO: `/v3/users/f51336aa2ca74094a000eb13016699bb` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:54.201530 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 885/1768] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:08:53 2022] DELETE /identity/v3/users/6c4e0c9a63074b3b87a0aff65e243589 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:54.206941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f51336aa2ca74094a000eb13016699bb)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:54.209716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:54.215226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90f705-a672-41d4-92b0-13e5784b0868 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:54.215372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90f705-a672-41d4-92b0-13e5784b0868 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:54.215662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90f705-a672-41d4-92b0-13e5784b0868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:54.308992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2b90f705-a672-41d4-92b0-13e5784b0868 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:54.312529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2b90f705-a672-41d4-92b0-13e5784b0868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:54.382939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f51336aa2ca74094a000eb13016699bb'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:54.384900 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f51336aa2ca74094a000eb13016699bb'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:54.405008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f51336aa2ca74094a000eb13016699bb'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:54.454980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f51336aa2ca74094a000eb13016699bb'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:54.458250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f51336aa2ca74094a000eb13016699bb'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:54.463241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:54.479742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 886/1769] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:54.520756 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 884/1770] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:08:54 2022] DELETE /identity/v3/users/f51336aa2ca74094a000eb13016699bb => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:54.525451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2f58b-1756-4b92-9a6a-93140480cb0d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:54.526547 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2f58b-1756-4b92-9a6a-93140480cb0d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:54.526999 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2f58b-1756-4b92-9a6a-93140480cb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:54.539516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:54.541593 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:54.587408 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-28b2f58b-1756-4b92-9a6a-93140480cb0d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:54.593101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-28b2f58b-1756-4b92-9a6a-93140480cb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:54.786255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b44096-8d5a-4885-8111-bc3ad125f9e5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:54.804137 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070dde4-c4c2-4457-a68e-7c82639f32d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:54.809242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070dde4-c4c2-4457-a68e-7c82639f32d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:54.809242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070dde4-c4c2-4457-a68e-7c82639f32d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:54.809242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070dde4-c4c2-4457-a68e-7c82639f32d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:54.886333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 887/1771] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:54.958765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2f58b-1756-4b92-9a6a-93140480cb0d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:54.975603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2f58b-1756-4b92-9a6a-93140480cb0d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:54.997284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6120dc4-6eae-4739-8eff-a30b16347204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:54.998007 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6120dc4-6eae-4739-8eff-a30b16347204 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:54.999179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6120dc4-6eae-4739-8eff-a30b16347204 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:54.999665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6120dc4-6eae-4739-8eff-a30b16347204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:55.013000 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:55.024679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6120dc4-6eae-4739-8eff-a30b16347204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:55.027810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6120dc4-6eae-4739-8eff-a30b16347204 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:55.057370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 888/1772] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:08:55.063712 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070dde4-c4c2-4457-a68e-7c82639f32d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:55.065634 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070dde4-c4c2-4457-a68e-7c82639f32d4 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:55.114102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 885/1773] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:55.626337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6120dc4-6eae-4739-8eff-a30b16347204 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:55.628001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c6120dc4-6eae-4739-8eff-a30b16347204 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:55.662564 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6120dc4-6eae-4739-8eff-a30b16347204 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:55.685028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5ed95-3668-4e95-89a8-1f5e4144c7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:55.686098 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5ed95-3668-4e95-89a8-1f5e4144c7f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:55.688354 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5ed95-3668-4e95-89a8-1f5e4144c7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:55.689386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5ed95-3668-4e95-89a8-1f5e4144c7f3 admin admin] PATH_INFO: `/v3/users/93a7eefdc67c4b798b491a6daff4b813` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:55.695932 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5ed95-3668-4e95-89a8-1f5e4144c7f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93a7eefdc67c4b798b491a6daff4b813)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:55.703972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5ed95-3668-4e95-89a8-1f5e4144c7f3 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:55.823423 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ce5ed95-3668-4e95-89a8-1f5e4144c7f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93a7eefdc67c4b798b491a6daff4b813'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:55.824997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ce5ed95-3668-4e95-89a8-1f5e4144c7f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93a7eefdc67c4b798b491a6daff4b813'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:55.843631 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ce5ed95-3668-4e95-89a8-1f5e4144c7f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93a7eefdc67c4b798b491a6daff4b813'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:55.866038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ce5ed95-3668-4e95-89a8-1f5e4144c7f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93a7eefdc67c4b798b491a6daff4b813'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:55.867065 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ce5ed95-3668-4e95-89a8-1f5e4144c7f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93a7eefdc67c4b798b491a6daff4b813'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:55.899732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36784610-c08a-4faf-818d-124d4fe27320 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:55.901122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36784610-c08a-4faf-818d-124d4fe27320 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:55.901122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36784610-c08a-4faf-818d-124d4fe27320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:55.901304 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 889/1774] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:08:55 2022] DELETE /identity/v3/users/93a7eefdc67c4b798b491a6daff4b813 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:55.951010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-36784610-c08a-4faf-818d-124d4fe27320 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:55.953765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-36784610-c08a-4faf-818d-124d4fe27320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:56.042539 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e4261-9429-447f-9bf0-74703e53e95e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:56.043008 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e4261-9429-447f-9bf0-74703e53e95e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:56.043479 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e4261-9429-447f-9bf0-74703e53e95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:56.114639 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a50e4261-9429-447f-9bf0-74703e53e95e None None] MFA Rules not processed for user `88d2351fc3b74ef5a530fded1be6b997`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:56.117344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a50e4261-9429-447f-9bf0-74703e53e95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:56.221591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 886/1775] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:08:56.251353 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36784610-c08a-4faf-818d-124d4fe27320 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:56.256089 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-36784610-c08a-4faf-818d-124d4fe27320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:56.358640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 890/1776] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:08:56.408775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:56.443837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36784610-c08a-4faf-818d-124d4fe27320 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:56.451344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e082d8-f86b-4958-a826-dac449f34873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:56.452127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e082d8-f86b-4958-a826-dac449f34873 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:56.452489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e082d8-f86b-4958-a826-dac449f34873 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:56.452773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e082d8-f86b-4958-a826-dac449f34873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:56.474521 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e082d8-f86b-4958-a826-dac449f34873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:56.480416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e082d8-f86b-4958-a826-dac449f34873 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:56.512266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 887/1777] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:56.604638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e4261-9429-447f-9bf0-74703e53e95e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:56.606474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a50e4261-9429-447f-9bf0-74703e53e95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:56.636981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e4261-9429-447f-9bf0-74703e53e95e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:56.647164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:56.648398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:56.648757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:56.649956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] PATH_INFO: `/v3/projects/d527c6b6d2a04c928f09d84af9449434` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:56.656676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d527c6b6d2a04c928f09d84af9449434)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:56.658933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:56.763116 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd527c6b6d2a04c928f09d84af9449434', 'request_id': 'req-50c5b855-300f-4ce5-9e2c-377da5338533'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:56.785453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd527c6b6d2a04c928f09d84af9449434', 'request_id': 'req-50c5b855-300f-4ce5-9e2c-377da5338533'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:56.804804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd527c6b6d2a04c928f09d84af9449434', 'request_id': 'req-50c5b855-300f-4ce5-9e2c-377da5338533'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:56.808375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:56.835013 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] The token cache is being invalidate because project d527c6b6d2a04c928f09d84af9449434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:08:56.835478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:56.837226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:56.839265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 891/1778] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:08:56 2022] DELETE /identity/v3/projects/d527c6b6d2a04c928f09d84af9449434 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:56.843442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e082d8-f86b-4958-a826-dac449f34873 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:56.859981 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e082d8-f86b-4958-a826-dac449f34873 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:57.044862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16830fc8-5e3d-4596-8e3f-5d2369cc49a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:57.045552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16830fc8-5e3d-4596-8e3f-5d2369cc49a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:57.045877 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16830fc8-5e3d-4596-8e3f-5d2369cc49a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:57.046149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16830fc8-5e3d-4596-8e3f-5d2369cc49a3 admin admin] PATH_INFO: `/v3/users/d7368028d4be4d20af118809edab2ffe` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:57.052072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16830fc8-5e3d-4596-8e3f-5d2369cc49a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7368028d4be4d20af118809edab2ffe)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:57.055195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16830fc8-5e3d-4596-8e3f-5d2369cc49a3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:57.192155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-16830fc8-5e3d-4596-8e3f-5d2369cc49a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7368028d4be4d20af118809edab2ffe'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:57.193513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-16830fc8-5e3d-4596-8e3f-5d2369cc49a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7368028d4be4d20af118809edab2ffe'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:57.211048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-16830fc8-5e3d-4596-8e3f-5d2369cc49a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7368028d4be4d20af118809edab2ffe'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:57.248022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-16830fc8-5e3d-4596-8e3f-5d2369cc49a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7368028d4be4d20af118809edab2ffe'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:57.249890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-16830fc8-5e3d-4596-8e3f-5d2369cc49a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7368028d4be4d20af118809edab2ffe'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:57.277983 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 888/1779] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:08:56 2022] DELETE /identity/v3/users/d7368028d4be4d20af118809edab2ffe => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:57.564032 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:57.567707 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:57.603595 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5b855-300f-4ce5-9e2c-377da5338533 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:57.782817 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3047dac-6ad2-4099-a8b0-c1921cfc5946 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:57.782966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3047dac-6ad2-4099-a8b0-c1921cfc5946 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:57.783344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3047dac-6ad2-4099-a8b0-c1921cfc5946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:57.793105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:57.801950 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:57.803579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:57.804515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] PATH_INFO: `/v3/projects/9a9e3f23bd3f4d489010a69e7f0906e4` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:57.808625 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a9e3f23bd3f4d489010a69e7f0906e4)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:57.813394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:57.850318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a3047dac-6ad2-4099-a8b0-c1921cfc5946 None None] MFA Rules not processed for user `3a1f11a4b8554a98b7626a82f01888c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:57.855653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a3047dac-6ad2-4099-a8b0-c1921cfc5946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:58.017837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a9e3f23bd3f4d489010a69e7f0906e4', 'request_id': 'req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:58.023657 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 889/1780] 10.222.0.74 () {58 vars in 1015 bytes} [Mon Jul 4 10:08:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:08:58.029724 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a9e3f23bd3f4d489010a69e7f0906e4', 'request_id': 'req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:58.039672 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a9e3f23bd3f4d489010a69e7f0906e4', 'request_id': 'req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:58.067609 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] The token cache is being invalidate because project 9a9e3f23bd3f4d489010a69e7f0906e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:08:58.067954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:58.069426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:58.073662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 892/1781] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:08:57 2022] DELETE /identity/v3/projects/9a9e3f23bd3f4d489010a69e7f0906e4 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:58.075896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3047dac-6ad2-4099-a8b0-c1921cfc5946 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:58.077504 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a3047dac-6ad2-4099-a8b0-c1921cfc5946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:58.108413 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3047dac-6ad2-4099-a8b0-c1921cfc5946 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:58.108639 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:58.110034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:58.166220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:58.254827 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8162e-1acf-4f23-8af8-c3ae9fa10eed admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:58.264867 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bec67-a36c-4706-a7e5-da880297dea7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:58.265567 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bec67-a36c-4706-a7e5-da880297dea7 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:58.265891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bec67-a36c-4706-a7e5-da880297dea7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:58.266167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bec67-a36c-4706-a7e5-da880297dea7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:58.290472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d69bec67-a36c-4706-a7e5-da880297dea7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:58.327203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:58.328130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:58.328422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:58.328699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] PATH_INFO: `/v3/users/e27cec66d1074697aeecebd635793923` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:58.329755 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bec67-a36c-4706-a7e5-da880297dea7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:58.330590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e27cec66d1074697aeecebd635793923)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:58.335015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:58.337337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bec67-a36c-4706-a7e5-da880297dea7 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:58.362909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 893/1782] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:58.374714 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bec67-a36c-4706-a7e5-da880297dea7 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:58.407964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bec67-a36c-4706-a7e5-da880297dea7 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:58.419964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5f9c2-29d6-44d3-ae72-f46a4bff0f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:58.420644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5f9c2-29d6-44d3-ae72-f46a4bff0f19 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:58.420936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5f9c2-29d6-44d3-ae72-f46a4bff0f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:58.421221 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5f9c2-29d6-44d3-ae72-f46a4bff0f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:58.445954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e27cec66d1074697aeecebd635793923'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:58.452057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5f9c2-29d6-44d3-ae72-f46a4bff0f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:58.453732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5f9c2-29d6-44d3-ae72-f46a4bff0f19 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:58.454144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e27cec66d1074697aeecebd635793923'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:58.475227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e27cec66d1074697aeecebd635793923'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:58.484948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 894/1783] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:08:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:08:58.512300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e27cec66d1074697aeecebd635793923'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:58.520770 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e27cec66d1074697aeecebd635793923'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:58.558359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 890/1784] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:08:58 2022] DELETE /identity/v3/users/e27cec66d1074697aeecebd635793923 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:58.973331 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5f9c2-29d6-44d3-ae72-f46a4bff0f19 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:58.975200 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4fe5f9c2-29d6-44d3-ae72-f46a4bff0f19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:59.002399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5f9c2-29d6-44d3-ae72-f46a4bff0f19 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:59.035114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:59.175091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:59.175945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:59.178446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:59.178797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] PATH_INFO: `/v3/projects/0ba469fa844c4bb0b80d3605ef36eec6` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:59.180972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba469fa844c4bb0b80d3605ef36eec6)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:59.186512 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:59.335809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba469fa844c4bb0b80d3605ef36eec6', 'request_id': 'req-3776b089-a507-4742-a8e7-ccc72ca60c23'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.352726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba469fa844c4bb0b80d3605ef36eec6', 'request_id': 'req-3776b089-a507-4742-a8e7-ccc72ca60c23'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.371216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba469fa844c4bb0b80d3605ef36eec6', 'request_id': 'req-3776b089-a507-4742-a8e7-ccc72ca60c23'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.389402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] The token cache is being invalidate because project 0ba469fa844c4bb0b80d3605ef36eec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:08:59.390038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.391575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.397941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 895/1785] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:08:58 2022] DELETE /identity/v3/projects/0ba469fa844c4bb0b80d3605ef36eec6 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:59.402056 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:59.404211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:59.433179 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64355b-7662-4ec1-8657-2cacbb2f7af7 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:59.567156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:08:59.568818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:59.573976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b60438-2830-4828-8f52-89eab8094c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:59.574915 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b60438-2830-4828-8f52-89eab8094c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:59.575207 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b60438-2830-4828-8f52-89eab8094c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:59.575493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b60438-2830-4828-8f52-89eab8094c3b admin admin] PATH_INFO: `/v3/users/278805d10f744e2f97d656fa6ef0abfe` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:59.577554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b60438-2830-4828-8f52-89eab8094c3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=278805d10f744e2f97d656fa6ef0abfe)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:59.579602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b60438-2830-4828-8f52-89eab8094c3b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:59.595354 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776b089-a507-4742-a8e7-ccc72ca60c23 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:08:59.607962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:08:59.610047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:59.613754 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:59.617285 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148 admin admin] PATH_INFO: `/v3/projects/570f0577f31f42c0b890084f308a7c5b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:59.622580 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=570f0577f31f42c0b890084f308a7c5b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:08:59.627233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:08:59.686231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:59.748445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a7b60438-2830-4828-8f52-89eab8094c3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '278805d10f744e2f97d656fa6ef0abfe'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.751454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a7b60438-2830-4828-8f52-89eab8094c3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '278805d10f744e2f97d656fa6ef0abfe'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.773158 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '570f0577f31f42c0b890084f308a7c5b', 'request_id': 'req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.781688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a7b60438-2830-4828-8f52-89eab8094c3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '278805d10f744e2f97d656fa6ef0abfe'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.803361 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '570f0577f31f42c0b890084f308a7c5b', 'request_id': 'req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.822717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a7b60438-2830-4828-8f52-89eab8094c3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '278805d10f744e2f97d656fa6ef0abfe'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.823217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '570f0577f31f42c0b890084f308a7c5b', 'request_id': 'req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.824501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a7b60438-2830-4828-8f52-89eab8094c3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '278805d10f744e2f97d656fa6ef0abfe'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.853532 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148 admin admin] The token cache is being invalidate because project 570f0577f31f42c0b890084f308a7c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:08:59.853890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.856297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f4784a1f-58c1-42e4-8cf9-56f57c6b3148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:08:59.864667 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 896/1786] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:08:59 2022] DELETE /identity/v3/projects/570f0577f31f42c0b890084f308a7c5b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:08:59.868566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 891/1787] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:08:59 2022] DELETE /identity/v3/users/278805d10f744e2f97d656fa6ef0abfe => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:08:59.868722 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eedeaa-30d5-4b07-bd9a-44d66f72868c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:59.868722 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eedeaa-30d5-4b07-bd9a-44d66f72868c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:59.868722 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eedeaa-30d5-4b07-bd9a-44d66f72868c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:08:59.893319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e0eedeaa-30d5-4b07-bd9a-44d66f72868c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:08:59.894834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e0eedeaa-30d5-4b07-bd9a-44d66f72868c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:08:59.929882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:08:59.951412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a2d9c-2fdc-45d4-a016-4d94f9ea09e0 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:08:59.951786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a2d9c-2fdc-45d4-a016-4d94f9ea09e0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:08:59.952067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a2d9c-2fdc-45d4-a016-4d94f9ea09e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:00.016664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d05a2d9c-2fdc-45d4-a016-4d94f9ea09e0 None None] MFA Rules not processed for user `1c05b9b08df0437f9b0abb167d17aa56`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:00.022361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d05a2d9c-2fdc-45d4-a016-4d94f9ea09e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:00.094184 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 897/1788] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:00.106660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a4ce0-da63-4b8d-8828-63af4a9f9621 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:00.110670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a4ce0-da63-4b8d-8828-63af4a9f9621 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:00.111907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a4ce0-da63-4b8d-8828-63af4a9f9621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:00.172300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b20a4ce0-da63-4b8d-8828-63af4a9f9621 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:00.178351 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b20a4ce0-da63-4b8d-8828-63af4a9f9621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:00.199749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 898/1789] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:00.201156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a4ce0-da63-4b8d-8828-63af4a9f9621 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:00.214554 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a4ce0-da63-4b8d-8828-63af4a9f9621 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:00.222077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454c8c3-71f2-488c-a2ca-b3c72abb7d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:00.223170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454c8c3-71f2-488c-a2ca-b3c72abb7d8b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:00.223623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454c8c3-71f2-488c-a2ca-b3c72abb7d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:00.224018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454c8c3-71f2-488c-a2ca-b3c72abb7d8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:00.224961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454c8c3-71f2-488c-a2ca-b3c72abb7d8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:09:00.226132 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454c8c3-71f2-488c-a2ca-b3c72abb7d8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:00.228744 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454c8c3-71f2-488c-a2ca-b3c72abb7d8b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:00.249436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 899/1790] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:09:00.250510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454c8c3-71f2-488c-a2ca-b3c72abb7d8b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:00.252831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 892/1791] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:08:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:00.264201 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454c8c3-71f2-488c-a2ca-b3c72abb7d8b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:00.271031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e0f37-10e9-4fb5-9dee-9929d21c6402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:00.272232 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e0f37-10e9-4fb5-9dee-9929d21c6402 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:00.272369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e0f37-10e9-4fb5-9dee-9929d21c6402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:00.272782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e0f37-10e9-4fb5-9dee-9929d21c6402 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:00.274104 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e0f37-10e9-4fb5-9dee-9929d21c6402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:00.276768 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e0f37-10e9-4fb5-9dee-9929d21c6402 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:00.321720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 900/1792] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:00 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:00.321908 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a2d9c-2fdc-45d4-a016-4d94f9ea09e0 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:00.336285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a2d9c-2fdc-45d4-a016-4d94f9ea09e0 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:00.424362 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5227d-de04-49f1-b386-202aab06bde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:00.430775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5227d-de04-49f1-b386-202aab06bde9 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:00.430775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5227d-de04-49f1-b386-202aab06bde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:00.431116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5227d-de04-49f1-b386-202aab06bde9 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:00.431862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5227d-de04-49f1-b386-202aab06bde9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:00.434809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5227d-de04-49f1-b386-202aab06bde9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:00.466845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 893/1793] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:00 2022] POST /identity/v3/users => generated 294 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:00.467072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e0f37-10e9-4fb5-9dee-9929d21c6402 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:00.487364 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e0f37-10e9-4fb5-9dee-9929d21c6402 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:00.495907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548692eb-6505-41db-ab82-c65b591c035a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:00.497081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548692eb-6505-41db-ab82-c65b591c035a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:00.497795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548692eb-6505-41db-ab82-c65b591c035a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:00.498553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548692eb-6505-41db-ab82-c65b591c035a admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:00.499833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548692eb-6505-41db-ab82-c65b591c035a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:00.502645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548692eb-6505-41db-ab82-c65b591c035a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:00.567889 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 901/1794] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:09:00.572224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5227d-de04-49f1-b386-202aab06bde9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:00.587563 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5227d-de04-49f1-b386-202aab06bde9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:00.606552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35e363-2a91-4d00-8466-f6994c5838c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:00.607252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35e363-2a91-4d00-8466-f6994c5838c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:00.607540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35e363-2a91-4d00-8466-f6994c5838c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:00.607804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35e363-2a91-4d00-8466-f6994c5838c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:00.608550 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35e363-2a91-4d00-8466-f6994c5838c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:00.621818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35e363-2a91-4d00-8466-f6994c5838c3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:00.669764 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 894/1795] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:00.671165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548692eb-6505-41db-ab82-c65b591c035a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:00.687921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548692eb-6505-41db-ab82-c65b591c035a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:00.701090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9966e01-4dcb-4440-98ea-2a43ac4b9a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:00.707829 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9966e01-4dcb-4440-98ea-2a43ac4b9a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:00.708200 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9966e01-4dcb-4440-98ea-2a43ac4b9a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:00.708491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9966e01-4dcb-4440-98ea-2a43ac4b9a5d admin admin] PATH_INFO: `/v3/projects/eda7dce32b864d139d83f66f7b4fed2f/users/7c60be904f864f139b58097ff7057fc3/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:00.770686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9966e01-4dcb-4440-98ea-2a43ac4b9a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda7dce32b864d139d83f66f7b4fed2f, user_id=7c60be904f864f139b58097ff7057fc3, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:00.778571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9966e01-4dcb-4440-98ea-2a43ac4b9a5d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:00.819713 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 902/1796] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:00 2022] PUT /identity/v3/projects/eda7dce32b864d139d83f66f7b4fed2f/users/7c60be904f864f139b58097ff7057fc3/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:09:00.834859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f450b2-641b-46fd-881f-208ea409a7c2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:00.836195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f450b2-641b-46fd-881f-208ea409a7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:00.836991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f450b2-641b-46fd-881f-208ea409a7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:00.906139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b2f450b2-641b-46fd-881f-208ea409a7c2 None None] MFA Rules not processed for user `7c60be904f864f139b58097ff7057fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:00.911524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b2f450b2-641b-46fd-881f-208ea409a7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:00.953983 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:01.073091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9966e01-4dcb-4440-98ea-2a43ac4b9a5d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:01.088044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 895/1797] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:01.116266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f450b2-641b-46fd-881f-208ea409a7c2 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:01.118327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b2f450b2-641b-46fd-881f-208ea409a7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:01.273673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9966e01-4dcb-4440-98ea-2a43ac4b9a5d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:01.286982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca394c-8c78-46a5-8cc8-7eb909da629c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:01.295876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca394c-8c78-46a5-8cc8-7eb909da629c admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:01.296257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca394c-8c78-46a5-8cc8-7eb909da629c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:01.297226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca394c-8c78-46a5-8cc8-7eb909da629c admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:01.301323 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca394c-8c78-46a5-8cc8-7eb909da629c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:01.303563 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca394c-8c78-46a5-8cc8-7eb909da629c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:01.322023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f450b2-641b-46fd-881f-208ea409a7c2 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:01.341884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22d88d-4a1b-4d57-8a5e-60888130aaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:01.342925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d88d-4a1b-4d57-8a5e-60888130aaef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:01.343298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d88d-4a1b-4d57-8a5e-60888130aaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:01.347654 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d88d-4a1b-4d57-8a5e-60888130aaef admin admin] PATH_INFO: `/v3/projects/bba58db694bb40458fdf4913147aa16a` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:01.350167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22d88d-4a1b-4d57-8a5e-60888130aaef admin admin] RBAC: Authorizing `identity:delete_project(project_id=bba58db694bb40458fdf4913147aa16a)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:01.353495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22d88d-4a1b-4d57-8a5e-60888130aaef admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:01.382984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 903/1798] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:01 2022] POST /identity/v3/projects => generated 387 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:01.387055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca394c-8c78-46a5-8cc8-7eb909da629c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:01.519229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4e22d88d-4a1b-4d57-8a5e-60888130aaef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bba58db694bb40458fdf4913147aa16a', 'request_id': 'req-4e22d88d-4a1b-4d57-8a5e-60888130aaef'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:01.523818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:01.533265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4e22d88d-4a1b-4d57-8a5e-60888130aaef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bba58db694bb40458fdf4913147aa16a', 'request_id': 'req-4e22d88d-4a1b-4d57-8a5e-60888130aaef'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:01.547738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4e22d88d-4a1b-4d57-8a5e-60888130aaef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bba58db694bb40458fdf4913147aa16a', 'request_id': 'req-4e22d88d-4a1b-4d57-8a5e-60888130aaef'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:01.577359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4e22d88d-4a1b-4d57-8a5e-60888130aaef admin admin] The token cache is being invalidate because project bba58db694bb40458fdf4913147aa16a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:01.578134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4e22d88d-4a1b-4d57-8a5e-60888130aaef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:01.579291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4e22d88d-4a1b-4d57-8a5e-60888130aaef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:01.583155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 896/1799] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:01 2022] DELETE /identity/v3/projects/bba58db694bb40458fdf4913147aa16a => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:01.585737 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca394c-8c78-46a5-8cc8-7eb909da629c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:01.588547 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f8d0c-ef33-4cc4-96c1-4675c618553f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:01.589069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f8d0c-ef33-4cc4-96c1-4675c618553f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:01.589658 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f8d0c-ef33-4cc4-96c1-4675c618553f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:01.631204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-666f8d0c-ef33-4cc4-96c1-4675c618553f None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:01.634396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-666f8d0c-ef33-4cc4-96c1-4675c618553f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:01.766544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c40c7-bcb3-4b4c-8576-54fbeefad0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:01.767637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c40c7-bcb3-4b4c-8576-54fbeefad0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:01.768028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c40c7-bcb3-4b4c-8576-54fbeefad0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:01.768417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c40c7-bcb3-4b4c-8576-54fbeefad0b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:01.769352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c40c7-bcb3-4b4c-8576-54fbeefad0b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:01.771838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c40c7-bcb3-4b4c-8576-54fbeefad0b5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:01.779350 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:01.812354 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 897/1800] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:01.814568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91f926-2ec8-41f4-af6b-da916f5e3b96 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:01.814864 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91f926-2ec8-41f4-af6b-da916f5e3b96 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:01.815878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91f926-2ec8-41f4-af6b-da916f5e3b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:01.821317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 904/1801] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:01 2022] POST /identity/v3/users => generated 300 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:01.825426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c40c7-bcb3-4b4c-8576-54fbeefad0b5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:01.828296 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1b0c40c7-bcb3-4b4c-8576-54fbeefad0b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:01.858684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4c91f926-2ec8-41f4-af6b-da916f5e3b96 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:01.860354 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4c91f926-2ec8-41f4-af6b-da916f5e3b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:01.861564 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c40c7-bcb3-4b4c-8576-54fbeefad0b5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:01.873004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 898/1802] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:01.877929 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d43308-48bf-4024-8536-6c618b92720a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:01.878317 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91f926-2ec8-41f4-af6b-da916f5e3b96 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:01.878882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d43308-48bf-4024-8536-6c618b92720a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:01.879269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d43308-48bf-4024-8536-6c618b92720a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:01.879652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d43308-48bf-4024-8536-6c618b92720a admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:01.880528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d43308-48bf-4024-8536-6c618b92720a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:01.882305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d43308-48bf-4024-8536-6c618b92720a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:01.890562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91f926-2ec8-41f4-af6b-da916f5e3b96 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:01.905203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 905/1803] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:01.909357 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d43308-48bf-4024-8536-6c618b92720a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:01.922236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a189152-f9a2-4ff7-ad4b-b81e3310308b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:01.923575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a189152-f9a2-4ff7-ad4b-b81e3310308b admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:01.925099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a189152-f9a2-4ff7-ad4b-b81e3310308b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:01.925944 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a189152-f9a2-4ff7-ad4b-b81e3310308b admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:01.929792 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a189152-f9a2-4ff7-ad4b-b81e3310308b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:09:01.929792 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a189152-f9a2-4ff7-ad4b-b81e3310308b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:01.929977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d43308-48bf-4024-8536-6c618b92720a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:01.931180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a189152-f9a2-4ff7-ad4b-b81e3310308b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:01.945910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa12518-046b-412a-8cb9-088d5ebf07fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:01.947024 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa12518-046b-412a-8cb9-088d5ebf07fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:01.947414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa12518-046b-412a-8cb9-088d5ebf07fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:01.947796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa12518-046b-412a-8cb9-088d5ebf07fe admin admin] PATH_INFO: `/v3/projects/170dad11fe8f4dc78ae64fc2cd79bf88/users/85a63cad2df546abac0bca6f94d4f305/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:01.967642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 899/1804] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:09:01.972051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a189152-f9a2-4ff7-ad4b-b81e3310308b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:01.983770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa12518-046b-412a-8cb9-088d5ebf07fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=170dad11fe8f4dc78ae64fc2cd79bf88, user_id=85a63cad2df546abac0bca6f94d4f305, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:01.988929 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa12518-046b-412a-8cb9-088d5ebf07fe admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:01.997776 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a189152-f9a2-4ff7-ad4b-b81e3310308b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:02.018564 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ebb98-539b-4472-989e-49ddb6309472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:02.020688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ebb98-539b-4472-989e-49ddb6309472 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:02.021127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 906/1805] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:01 2022] PUT /identity/v3/projects/170dad11fe8f4dc78ae64fc2cd79bf88/users/85a63cad2df546abac0bca6f94d4f305/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:02.021958 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ebb98-539b-4472-989e-49ddb6309472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:02.023194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ebb98-539b-4472-989e-49ddb6309472 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:02.024708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ebb98-539b-4472-989e-49ddb6309472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:02.025212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa12518-046b-412a-8cb9-088d5ebf07fe admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:02.031609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ebb98-539b-4472-989e-49ddb6309472 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:02.109996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 900/1806] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:01 2022] POST /identity/v3/projects => generated 407 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:02.113054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ebb98-539b-4472-989e-49ddb6309472 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:02.175975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa12518-046b-412a-8cb9-088d5ebf07fe admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:02.292656 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:02.301569 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ebb98-539b-4472-989e-49ddb6309472 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:02.316802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aed725-cc29-4dff-88e8-f27cf15fd652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:02.319435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aed725-cc29-4dff-88e8-f27cf15fd652 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:02.322159 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aed725-cc29-4dff-88e8-f27cf15fd652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:02.322739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aed725-cc29-4dff-88e8-f27cf15fd652 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:02.326792 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aed725-cc29-4dff-88e8-f27cf15fd652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:02.329697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aed725-cc29-4dff-88e8-f27cf15fd652 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:02.341150 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d942b2f-c291-4550-ab1d-c1a5c7824218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:02.341941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d942b2f-c291-4550-ab1d-c1a5c7824218 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:02.342229 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d942b2f-c291-4550-ab1d-c1a5c7824218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:02.342529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d942b2f-c291-4550-ab1d-c1a5c7824218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:02.343279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d942b2f-c291-4550-ab1d-c1a5c7824218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:02.355544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d942b2f-c291-4550-ab1d-c1a5c7824218 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:02.394829 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 907/1807] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:02.395961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 901/1808] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:02 2022] POST /identity/v3/users => generated 304 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:02.397676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d942b2f-c291-4550-ab1d-c1a5c7824218 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:02.406045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aed725-cc29-4dff-88e8-f27cf15fd652 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:02.430057 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aed725-cc29-4dff-88e8-f27cf15fd652 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:02.431496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d942b2f-c291-4550-ab1d-c1a5c7824218 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:02.440100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a66e32-7815-4e6c-b7b3-ea044f63561b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:02.441222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a66e32-7815-4e6c-b7b3-ea044f63561b admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:02.441507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a66e32-7815-4e6c-b7b3-ea044f63561b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:02.441827 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a66e32-7815-4e6c-b7b3-ea044f63561b admin admin] PATH_INFO: `/v3/projects/170dad11fe8f4dc78ae64fc2cd79bf88/users/85a63cad2df546abac0bca6f94d4f305/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:02.443756 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f93f53-92e9-4c39-9250-fad32bbdffd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:02.444469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f93f53-92e9-4c39-9250-fad32bbdffd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:02.447811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f93f53-92e9-4c39-9250-fad32bbdffd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:02.448572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f93f53-92e9-4c39-9250-fad32bbdffd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:02.450710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f93f53-92e9-4c39-9250-fad32bbdffd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:02.456806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a66e32-7815-4e6c-b7b3-ea044f63561b admin admin] RBAC: Authorizing `identity:create_grant(project_id=170dad11fe8f4dc78ae64fc2cd79bf88, user_id=85a63cad2df546abac0bca6f94d4f305, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:02.458476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f93f53-92e9-4c39-9250-fad32bbdffd6 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:02.465011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a66e32-7815-4e6c-b7b3-ea044f63561b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:02.489939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 902/1809] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:02.491462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d8bbc-e032-4a93-911d-170443382f7c None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:02.491780 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d8bbc-e032-4a93-911d-170443382f7c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:02.492059 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d8bbc-e032-4a93-911d-170443382f7c None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:02.494143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 903/1810] 10.222.0.79 () {54 vars in 921 bytes} [Mon Jul 4 10:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:09:02.497937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 908/1811] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:02 2022] PUT /identity/v3/projects/170dad11fe8f4dc78ae64fc2cd79bf88/users/85a63cad2df546abac0bca6f94d4f305/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:09:02.499164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d8bbc-e032-4a93-911d-170443382f7c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:02.510142 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838607a-e443-4554-ac04-eb43440251d1 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:02.510454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838607a-e443-4554-ac04-eb43440251d1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:02.510736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838607a-e443-4554-ac04-eb43440251d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:02.575618 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7838607a-e443-4554-ac04-eb43440251d1 None None] MFA Rules not processed for user `85a63cad2df546abac0bca6f94d4f305`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:02.579095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7838607a-e443-4554-ac04-eb43440251d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:02.762204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 909/1812] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:09:02.764106 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336a984-303c-42e1-9e9c-fd81b4d45202 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:02.764524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336a984-303c-42e1-9e9c-fd81b4d45202 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:02.764892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336a984-303c-42e1-9e9c-fd81b4d45202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:02.784496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d8bbc-e032-4a93-911d-170443382f7c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:02.790988 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c918371-86f5-491d-9e3d-24203956c502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:02.791680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c918371-86f5-491d-9e3d-24203956c502 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:02.791970 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c918371-86f5-491d-9e3d-24203956c502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:02.792251 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c918371-86f5-491d-9e3d-24203956c502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:02.793095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c918371-86f5-491d-9e3d-24203956c502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:02.799459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c918371-86f5-491d-9e3d-24203956c502 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:02.824157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4336a984-303c-42e1-9e9c-fd81b4d45202 None None] MFA Rules not processed for user `ffc2409cb7fc4373a74ddfb48d501d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:02.826006 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 904/1813] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:02.826464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4336a984-303c-42e1-9e9c-fd81b4d45202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:02.826979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c918371-86f5-491d-9e3d-24203956c502 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:02.828829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1c918371-86f5-491d-9e3d-24203956c502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:02.876009 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:02.958144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c918371-86f5-491d-9e3d-24203956c502 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:02.970943 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4223ae7-4f50-4c59-ae18-21c3fb582af3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:02.972781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4223ae7-4f50-4c59-ae18-21c3fb582af3 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:02.972781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4223ae7-4f50-4c59-ae18-21c3fb582af3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:02.974229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4223ae7-4f50-4c59-ae18-21c3fb582af3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:03.012523 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4223ae7-4f50-4c59-ae18-21c3fb582af3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:03.016052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4223ae7-4f50-4c59-ae18-21c3fb582af3 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:03.047673 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 905/1814] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:03.048026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 910/1815] 10.222.0.79 () {58 vars in 1018 bytes} [Mon Jul 4 10:09:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:03.049901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336a984-303c-42e1-9e9c-fd81b4d45202 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:03.084512 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336a984-303c-42e1-9e9c-fd81b4d45202 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:03.109189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5267ed2-5d27-4b8c-8b78-7d3fda189d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:03.126873 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5267ed2-5d27-4b8c-8b78-7d3fda189d71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:03.126873 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5267ed2-5d27-4b8c-8b78-7d3fda189d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:03.128283 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5267ed2-5d27-4b8c-8b78-7d3fda189d71 admin admin] PATH_INFO: `/v3/projects/d92112fc41964271b6f085fdb7df7f99/users/b6437457fb0a4512bcdd8994cfa6af75/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:03.189727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5267ed2-5d27-4b8c-8b78-7d3fda189d71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92112fc41964271b6f085fdb7df7f99, user_id=b6437457fb0a4512bcdd8994cfa6af75, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:03.200769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5267ed2-5d27-4b8c-8b78-7d3fda189d71 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:03.264193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ed0a1-a7de-4ed8-a4ee-6d34337615bd None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:03.264328 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 911/1816] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:03 2022] PUT /identity/v3/projects/d92112fc41964271b6f085fdb7df7f99/users/b6437457fb0a4512bcdd8994cfa6af75/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:03.264435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ed0a1-a7de-4ed8-a4ee-6d34337615bd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:03.264435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ed0a1-a7de-4ed8-a4ee-6d34337615bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:03.382479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-737ed0a1-a7de-4ed8-a4ee-6d34337615bd None None] MFA Rules not processed for user `b6437457fb0a4512bcdd8994cfa6af75`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:03.383909 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-737ed0a1-a7de-4ed8-a4ee-6d34337615bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:03.389497 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5267ed2-5d27-4b8c-8b78-7d3fda189d71 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:03.467580 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:03.580686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5267ed2-5d27-4b8c-8b78-7d3fda189d71 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:03.605868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f2aaa-7e73-4c80-b595-3c7585d57705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:03.608303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f2aaa-7e73-4c80-b595-3c7585d57705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:03.610684 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f2aaa-7e73-4c80-b595-3c7585d57705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:03.613183 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f2aaa-7e73-4c80-b595-3c7585d57705 admin admin] PATH_INFO: `/v3/users/7c60be904f864f139b58097ff7057fc3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:03.623436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f2aaa-7e73-4c80-b595-3c7585d57705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c60be904f864f139b58097ff7057fc3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:03.627836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f2aaa-7e73-4c80-b595-3c7585d57705 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:03.665768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 906/1817] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:03 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:03.669968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ed0a1-a7de-4ed8-a4ee-6d34337615bd None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:03.711225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ed0a1-a7de-4ed8-a4ee-6d34337615bd None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:03.727783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736def6-2c09-4da7-883a-9ab1c9d2b5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:03.734918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736def6-2c09-4da7-883a-9ab1c9d2b5dd admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:03.734918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736def6-2c09-4da7-883a-9ab1c9d2b5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:03.734918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736def6-2c09-4da7-883a-9ab1c9d2b5dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:03.734918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736def6-2c09-4da7-883a-9ab1c9d2b5dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:03.743684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736def6-2c09-4da7-883a-9ab1c9d2b5dd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:03.786930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-670f2aaa-7e73-4c80-b595-3c7585d57705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c60be904f864f139b58097ff7057fc3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:03.791539 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-670f2aaa-7e73-4c80-b595-3c7585d57705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c60be904f864f139b58097ff7057fc3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:03.815612 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-670f2aaa-7e73-4c80-b595-3c7585d57705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c60be904f864f139b58097ff7057fc3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:03.899671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 907/1818] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:03 2022] POST /identity/v3/projects => generated 409 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:03.900085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-670f2aaa-7e73-4c80-b595-3c7585d57705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c60be904f864f139b58097ff7057fc3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:03.900085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-670f2aaa-7e73-4c80-b595-3c7585d57705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c60be904f864f139b58097ff7057fc3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:03.900758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736def6-2c09-4da7-883a-9ab1c9d2b5dd admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:03.938927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6736def6-2c09-4da7-883a-9ab1c9d2b5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:03.984758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 912/1819] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:03 2022] DELETE /identity/v3/users/7c60be904f864f139b58097ff7057fc3 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:09:03.990011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dca4b-033f-41d4-be85-563048944388 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:03.990308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dca4b-033f-41d4-be85-563048944388 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:03.990591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dca4b-033f-41d4-be85-563048944388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:04.057494 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b65dca4b-033f-41d4-be85-563048944388 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:04.058878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b65dca4b-033f-41d4-be85-563048944388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:04.190968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736def6-2c09-4da7-883a-9ab1c9d2b5dd admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:04.238209 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 913/1820] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:04.287075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dca4b-033f-41d4-be85-563048944388 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:04.289473 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b65dca4b-033f-41d4-be85-563048944388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:04.331676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:04.360220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd6407-dfc1-456a-a6c0-facbbd3b5970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:04.361055 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd6407-dfc1-456a-a6c0-facbbd3b5970 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:04.361960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd6407-dfc1-456a-a6c0-facbbd3b5970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:04.362379 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd6407-dfc1-456a-a6c0-facbbd3b5970 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:04.363097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd6407-dfc1-456a-a6c0-facbbd3b5970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:04.392958 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd6407-dfc1-456a-a6c0-facbbd3b5970 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:04.491300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:04.510682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 908/1821] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:03 2022] POST /identity/v3/users => generated 311 bytes in 609 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:04.511865 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd6407-dfc1-456a-a6c0-facbbd3b5970 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:04.515728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e6bd6407-dfc1-456a-a6c0-facbbd3b5970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:04.653036 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dca4b-033f-41d4-be85-563048944388 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:04.687584 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04b4e3-8e24-4183-b64c-fd9c48cd37db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:04.690955 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04b4e3-8e24-4183-b64c-fd9c48cd37db service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:04.694863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04b4e3-8e24-4183-b64c-fd9c48cd37db service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:04.697537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04b4e3-8e24-4183-b64c-fd9c48cd37db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:04.707951 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ac04b4e3-8e24-4183-b64c-fd9c48cd37db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:04.715910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd6407-dfc1-456a-a6c0-facbbd3b5970 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:04.727631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f5120-1575-4955-adb0-beb751bc9d5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:04.728489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f5120-1575-4955-adb0-beb751bc9d5c service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:04.728854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f5120-1575-4955-adb0-beb751bc9d5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:04.729330 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f5120-1575-4955-adb0-beb751bc9d5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:04.753750 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f5120-1575-4955-adb0-beb751bc9d5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:04.755374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f5120-1575-4955-adb0-beb751bc9d5c service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:04.788139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 909/1822] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:04.790198 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04b4e3-8e24-4183-b64c-fd9c48cd37db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:04.792037 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04b4e3-8e24-4183-b64c-fd9c48cd37db service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:04.807084 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f5120-1575-4955-adb0-beb751bc9d5c service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:04.826338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f5120-1575-4955-adb0-beb751bc9d5c service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:04.845687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 914/1823] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:09:04.848941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04b4e3-8e24-4183-b64c-fd9c48cd37db service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:04.853147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ac04b4e3-8e24-4183-b64c-fd9c48cd37db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:04.858956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d1c8a-d7db-45f9-a5c0-08fa5066a210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:04.860378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d1c8a-d7db-45f9-a5c0-08fa5066a210 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:04.864328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d1c8a-d7db-45f9-a5c0-08fa5066a210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:04.868422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d1c8a-d7db-45f9-a5c0-08fa5066a210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:04.869374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d1c8a-d7db-45f9-a5c0-08fa5066a210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:04.873294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d1c8a-d7db-45f9-a5c0-08fa5066a210 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:04.915834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04b4e3-8e24-4183-b64c-fd9c48cd37db service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:04.919278 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 910/1824] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:04.923870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d1c8a-d7db-45f9-a5c0-08fa5066a210 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:04.942496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:04.947573 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:04.947979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:04.948359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071 admin admin] PATH_INFO: `/v3/projects/e5ad7ae2e74e40e89ba11f30d7089a2b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:04.949874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5ad7ae2e74e40e89ba11f30d7089a2b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:04.953321 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:04.960623 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d1c8a-d7db-45f9-a5c0-08fa5066a210 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:04.990476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ec4d1-385d-44b3-a6e0-2a5b24b9a212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:04.993663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ec4d1-385d-44b3-a6e0-2a5b24b9a212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:04.993805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ec4d1-385d-44b3-a6e0-2a5b24b9a212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:04.995252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ec4d1-385d-44b3-a6e0-2a5b24b9a212 admin admin] PATH_INFO: `/v3/projects/99e7eaca5ea249339279ea246ef84f5a/users/ca03cf56b1744229941ade383eddef24/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:05.065332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ec4d1-385d-44b3-a6e0-2a5b24b9a212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e7eaca5ea249339279ea246ef84f5a, user_id=ca03cf56b1744229941ade383eddef24, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:05.068341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ec4d1-385d-44b3-a6e0-2a5b24b9a212 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:05.114842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 911/1825] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:04 2022] PUT /identity/v3/projects/99e7eaca5ea249339279ea246ef84f5a/users/ca03cf56b1744229941ade383eddef24/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:05.115749 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ec4d1-385d-44b3-a6e0-2a5b24b9a212 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:05.143925 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5ad7ae2e74e40e89ba11f30d7089a2b', 'request_id': 'req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:05.171927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5ad7ae2e74e40e89ba11f30d7089a2b', 'request_id': 'req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:05.198063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5ad7ae2e74e40e89ba11f30d7089a2b', 'request_id': 'req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:05.219579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071 admin admin] The token cache is being invalidate because project e5ad7ae2e74e40e89ba11f30d7089a2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:05.221402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:05.221913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-955bf4ce-b9c5-4c93-b9dd-e47f5f35a071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:05.227928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 915/1826] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:04 2022] DELETE /identity/v3/projects/e5ad7ae2e74e40e89ba11f30d7089a2b => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:05.229498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d420db-6aa4-44af-b8cd-2c155169b2e7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:05.231786 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d420db-6aa4-44af-b8cd-2c155169b2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:05.232929 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d420db-6aa4-44af-b8cd-2c155169b2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:05.286811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a9d420db-6aa4-44af-b8cd-2c155169b2e7 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:05.288172 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a9d420db-6aa4-44af-b8cd-2c155169b2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:05.296346 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ec4d1-385d-44b3-a6e0-2a5b24b9a212 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:05.427536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 916/1827] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:05.436314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6ab0f-535d-46c2-ad79-c8d6cafa0ce4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:05.436734 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6ab0f-535d-46c2-ad79-c8d6cafa0ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:05.437108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6ab0f-535d-46c2-ad79-c8d6cafa0ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:05.462778 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:05.485522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-28d6ab0f-535d-46c2-ad79-c8d6cafa0ce4 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:05.487146 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-28d6ab0f-535d-46c2-ad79-c8d6cafa0ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:05.502614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23ff56-7e16-4f88-b392-e199db648e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:05.505962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23ff56-7e16-4f88-b392-e199db648e61 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:05.509333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23ff56-7e16-4f88-b392-e199db648e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:05.512265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 917/1828] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:09:05.514162 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23ff56-7e16-4f88-b392-e199db648e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:05.516338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23ff56-7e16-4f88-b392-e199db648e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:05.516695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6ab0f-535d-46c2-ad79-c8d6cafa0ce4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:05.527739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23ff56-7e16-4f88-b392-e199db648e61 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:05.544999 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6ab0f-535d-46c2-ad79-c8d6cafa0ce4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:05.557140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 912/1829] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:05.558931 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23ff56-7e16-4f88-b392-e199db648e61 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:05.561859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889219a0-6e58-45f7-ab57-c7cc6f053184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:05.563167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889219a0-6e58-45f7-ab57-c7cc6f053184 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:05.567534 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889219a0-6e58-45f7-ab57-c7cc6f053184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:05.568596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889219a0-6e58-45f7-ab57-c7cc6f053184 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:05.569082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6e23ff56-7e16-4f88-b392-e199db648e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:05.570038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889219a0-6e58-45f7-ab57-c7cc6f053184 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:09:05.571920 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889219a0-6e58-45f7-ab57-c7cc6f053184 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:05.573731 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889219a0-6e58-45f7-ab57-c7cc6f053184 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:05.617856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23ff56-7e16-4f88-b392-e199db648e61 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:05.635148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 918/1830] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:09:05.638849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1c186-5be0-42af-86ff-71d08e994a7c None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:05.639050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1c186-5be0-42af-86ff-71d08e994a7c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:05.639824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1c186-5be0-42af-86ff-71d08e994a7c None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:05.642149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8de43-c40d-46ee-ae4c-bcaea23c5039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:05.644479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8de43-c40d-46ee-ae4c-bcaea23c5039 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:05.644807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8de43-c40d-46ee-ae4c-bcaea23c5039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:05.645060 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8de43-c40d-46ee-ae4c-bcaea23c5039 admin admin] PATH_INFO: `/v3/projects/99e7eaca5ea249339279ea246ef84f5a/users/ca03cf56b1744229941ade383eddef24/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:05.645984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 919/1831] 10.222.0.79 () {54 vars in 921 bytes} [Mon Jul 4 10:09:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:09:05.646857 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1c186-5be0-42af-86ff-71d08e994a7c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:05.649808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8de43-c40d-46ee-ae4c-bcaea23c5039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e7eaca5ea249339279ea246ef84f5a, user_id=ca03cf56b1744229941ade383eddef24, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:05.659435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8de43-c40d-46ee-ae4c-bcaea23c5039 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:05.682136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1c186-5be0-42af-86ff-71d08e994a7c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:05.702237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 913/1832] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:05 2022] PUT /identity/v3/projects/99e7eaca5ea249339279ea246ef84f5a/users/ca03cf56b1744229941ade383eddef24/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:05.706809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561d98e-c235-4658-ab23-b436a0e6ce09 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:05.707016 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561d98e-c235-4658-ab23-b436a0e6ce09 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:05.707333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561d98e-c235-4658-ab23-b436a0e6ce09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:05.766581 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:05.797514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f561d98e-c235-4658-ab23-b436a0e6ce09 None None] MFA Rules not processed for user `ca03cf56b1744229941ade383eddef24`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:05.798851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f561d98e-c235-4658-ab23-b436a0e6ce09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:05.879587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293e819-2109-4ee9-870a-adc27b142fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:05.880289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293e819-2109-4ee9-870a-adc27b142fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:05.880574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293e819-2109-4ee9-870a-adc27b142fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:05.880856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293e819-2109-4ee9-870a-adc27b142fc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:05.881561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293e819-2109-4ee9-870a-adc27b142fc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:05.883554 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293e819-2109-4ee9-870a-adc27b142fc5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:05.948017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 920/1833] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:05 2022] POST /identity/v3/projects => generated 399 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:05.952764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b6e5b-df92-447b-b5e0-c0cb31e37e07 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:05.952764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b6e5b-df92-447b-b5e0-c0cb31e37e07 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:05.953096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b6e5b-df92-447b-b5e0-c0cb31e37e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:06.001188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-195b6e5b-df92-447b-b5e0-c0cb31e37e07 None None] MFA Rules not processed for user `ffc2409cb7fc4373a74ddfb48d501d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:06.002616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-195b6e5b-df92-447b-b5e0-c0cb31e37e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:06.010699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 914/1834] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:05 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:06.011599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561d98e-c235-4658-ab23-b436a0e6ce09 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:06.036443 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561d98e-c235-4658-ab23-b436a0e6ce09 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:06.210083 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 921/1835] 10.222.0.79 () {58 vars in 1018 bytes} [Mon Jul 4 10:09:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:06.213026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b6e5b-df92-447b-b5e0-c0cb31e37e07 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:06.214276 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14a3a6-bf69-41f1-8b67-73864c2b062c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:06.216755 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-195b6e5b-df92-447b-b5e0-c0cb31e37e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:06.218697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14a3a6-bf69-41f1-8b67-73864c2b062c admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:06.218987 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14a3a6-bf69-41f1-8b67-73864c2b062c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:06.219262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14a3a6-bf69-41f1-8b67-73864c2b062c admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:06.220359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14a3a6-bf69-41f1-8b67-73864c2b062c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:06.222574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14a3a6-bf69-41f1-8b67-73864c2b062c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:06.301049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 915/1836] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:06 2022] POST /identity/v3/users => generated 300 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:06.304707 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14a3a6-bf69-41f1-8b67-73864c2b062c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:06.344371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14a3a6-bf69-41f1-8b67-73864c2b062c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:06.390595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ae7c5-7c1b-4796-aaf0-80318fa94a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:06.394426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b6e5b-df92-447b-b5e0-c0cb31e37e07 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:06.395925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ae7c5-7c1b-4796-aaf0-80318fa94a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:06.396649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ae7c5-7c1b-4796-aaf0-80318fa94a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:06.396997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ae7c5-7c1b-4796-aaf0-80318fa94a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:06.397757 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ae7c5-7c1b-4796-aaf0-80318fa94a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:06.399764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356e5c0-da4b-4655-807b-3b3f89a6da61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:06.400594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356e5c0-da4b-4655-807b-3b3f89a6da61 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:06.401038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356e5c0-da4b-4655-807b-3b3f89a6da61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:06.401459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356e5c0-da4b-4655-807b-3b3f89a6da61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:06.405252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ae7c5-7c1b-4796-aaf0-80318fa94a19 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:06.452786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 916/1837] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:06.458368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ae7c5-7c1b-4796-aaf0-80318fa94a19 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:06.484195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356e5c0-da4b-4655-807b-3b3f89a6da61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:06.484195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356e5c0-da4b-4655-807b-3b3f89a6da61 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:06.489197 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:06.513023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ae7c5-7c1b-4796-aaf0-80318fa94a19 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:06.516855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 922/1838] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:06 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:06.521381 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d815880-c07d-41f4-a488-3c78addd982d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:06.522412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d815880-c07d-41f4-a488-3c78addd982d admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:06.523011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d815880-c07d-41f4-a488-3c78addd982d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:06.523813 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d815880-c07d-41f4-a488-3c78addd982d admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:06.525012 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d815880-c07d-41f4-a488-3c78addd982d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:06.527169 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d815880-c07d-41f4-a488-3c78addd982d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:06.541511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 917/1839] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:09:06.547227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356e5c0-da4b-4655-807b-3b3f89a6da61 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:06.590914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356e5c0-da4b-4655-807b-3b3f89a6da61 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:06.619443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64315dd2-f6f5-4a80-9457-064060ce6806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:06.623697 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64315dd2-f6f5-4a80-9457-064060ce6806 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:06.624049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64315dd2-f6f5-4a80-9457-064060ce6806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:06.624290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64315dd2-f6f5-4a80-9457-064060ce6806 admin admin] PATH_INFO: `/v3/projects/7b726e6855e44839a1c1abeee4d7bdc1/users/291be646e79641dfa03cf3eb41d39bad/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:06.663151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64315dd2-f6f5-4a80-9457-064060ce6806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b726e6855e44839a1c1abeee4d7bdc1, user_id=291be646e79641dfa03cf3eb41d39bad, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:06.666739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64315dd2-f6f5-4a80-9457-064060ce6806 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:06.705610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 923/1840] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:06 2022] PUT /identity/v3/projects/7b726e6855e44839a1c1abeee4d7bdc1/users/291be646e79641dfa03cf3eb41d39bad/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:06.716962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b285c-4c0a-4535-8ee4-8ded43abca97 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:06.717341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b285c-4c0a-4535-8ee4-8ded43abca97 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:06.717523 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b285c-4c0a-4535-8ee4-8ded43abca97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:06.931436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64315dd2-f6f5-4a80-9457-064060ce6806 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:07.071410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64315dd2-f6f5-4a80-9457-064060ce6806 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:07.083210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15e9e9-b34d-4692-bec5-b9477c371eee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:07.087228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15e9e9-b34d-4692-bec5-b9477c371eee service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:07.088180 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15e9e9-b34d-4692-bec5-b9477c371eee service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:07.089769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15e9e9-b34d-4692-bec5-b9477c371eee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:07.095638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ff15e9e9-b34d-4692-bec5-b9477c371eee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:07.119527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-0e9b285c-4c0a-4535-8ee4-8ded43abca97 None None] MFA Rules not processed for user `291be646e79641dfa03cf3eb41d39bad`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:07.136093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0e9b285c-4c0a-4535-8ee4-8ded43abca97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:07.158960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:07.225981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15e9e9-b34d-4692-bec5-b9477c371eee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:07.227476 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15e9e9-b34d-4692-bec5-b9477c371eee service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:07.239435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 924/1841] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:06 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:07.257104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 918/1842] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 539 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:07.258653 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15e9e9-b34d-4692-bec5-b9477c371eee service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:07.348794 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15e9e9-b34d-4692-bec5-b9477c371eee service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:07.354657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403375c8-55f1-40cf-b859-494950492ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:07.355616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403375c8-55f1-40cf-b859-494950492ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:07.356016 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403375c8-55f1-40cf-b859-494950492ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:07.356313 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403375c8-55f1-40cf-b859-494950492ce4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:07.357136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403375c8-55f1-40cf-b859-494950492ce4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:07.359887 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403375c8-55f1-40cf-b859-494950492ce4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:07.404043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 925/1843] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:07 2022] POST /identity/v3/projects => generated 399 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:09:07.409817 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b285c-4c0a-4535-8ee4-8ded43abca97 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:07.426119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b285c-4c0a-4535-8ee4-8ded43abca97 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:07.567597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b0945-4eda-4238-9b83-e5215db55fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:07.569076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b0945-4eda-4238-9b83-e5215db55fee admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:07.569368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b0945-4eda-4238-9b83-e5215db55fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:07.569665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b0945-4eda-4238-9b83-e5215db55fee admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:07.570770 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b0945-4eda-4238-9b83-e5215db55fee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:07.576733 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b0945-4eda-4238-9b83-e5215db55fee admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:07.640540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 919/1844] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:07 2022] POST /identity/v3/users => generated 306 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:07.644923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403375c8-55f1-40cf-b859-494950492ce4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:07.660669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403375c8-55f1-40cf-b859-494950492ce4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:07.681426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f211cbe-bf24-4740-b1de-6fc212e80f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:07.682175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f211cbe-bf24-4740-b1de-6fc212e80f86 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:07.682454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f211cbe-bf24-4740-b1de-6fc212e80f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:07.682726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f211cbe-bf24-4740-b1de-6fc212e80f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:07.683617 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f211cbe-bf24-4740-b1de-6fc212e80f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:07.685263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f211cbe-bf24-4740-b1de-6fc212e80f86 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:07.711498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:07.726144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b0945-4eda-4238-9b83-e5215db55fee admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:07.729530 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 926/1845] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:07.740391 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b0945-4eda-4238-9b83-e5215db55fee admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:07.746537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802955f3-8a32-415c-88b5-e1a541dd982c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:07.747723 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802955f3-8a32-415c-88b5-e1a541dd982c admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:07.748233 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802955f3-8a32-415c-88b5-e1a541dd982c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:07.748731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802955f3-8a32-415c-88b5-e1a541dd982c admin admin] PATH_INFO: `/v3/projects/0efaf2a4ac154b5ea2c1de047312d66f/users/8345647e62b64a3e9fe27374209b5722/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:07.770480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802955f3-8a32-415c-88b5-e1a541dd982c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0efaf2a4ac154b5ea2c1de047312d66f, user_id=8345647e62b64a3e9fe27374209b5722, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:07.774878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802955f3-8a32-415c-88b5-e1a541dd982c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:07.799757 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 920/1846] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:07 2022] PUT /identity/v3/projects/0efaf2a4ac154b5ea2c1de047312d66f/users/8345647e62b64a3e9fe27374209b5722/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:07.803531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f211cbe-bf24-4740-b1de-6fc212e80f86 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:07.816345 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f211cbe-bf24-4740-b1de-6fc212e80f86 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:07.931319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a4af7-fff1-4f05-9cc1-31e5c8a037d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:07.931932 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a4af7-fff1-4f05-9cc1-31e5c8a037d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:07.932229 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a4af7-fff1-4f05-9cc1-31e5c8a037d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:07.935207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a4af7-fff1-4f05-9cc1-31e5c8a037d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:07.936030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a4af7-fff1-4f05-9cc1-31e5c8a037d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:07.937758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a4af7-fff1-4f05-9cc1-31e5c8a037d1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:07.952679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 927/1847] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:07.953941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802955f3-8a32-415c-88b5-e1a541dd982c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:07.965046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802955f3-8a32-415c-88b5-e1a541dd982c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:07.975459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402404a5-c7e2-4839-bf49-efa7ed34eb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:07.977465 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402404a5-c7e2-4839-bf49-efa7ed34eb9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:07.977886 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402404a5-c7e2-4839-bf49-efa7ed34eb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:07.978377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402404a5-c7e2-4839-bf49-efa7ed34eb9b admin admin] PATH_INFO: `/v3/projects/0efaf2a4ac154b5ea2c1de047312d66f/users/8345647e62b64a3e9fe27374209b5722/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:07.981495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402404a5-c7e2-4839-bf49-efa7ed34eb9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0efaf2a4ac154b5ea2c1de047312d66f, user_id=8345647e62b64a3e9fe27374209b5722, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:07.984990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402404a5-c7e2-4839-bf49-efa7ed34eb9b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:07.995595 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a4af7-fff1-4f05-9cc1-31e5c8a037d1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:07.998083 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-eb0a4af7-fff1-4f05-9cc1-31e5c8a037d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:08.013675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 921/1848] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:07 2022] PUT /identity/v3/projects/0efaf2a4ac154b5ea2c1de047312d66f/users/8345647e62b64a3e9fe27374209b5722/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:08.023036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf4246-3d57-4af0-8520-979dc426cba0 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:08.023566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf4246-3d57-4af0-8520-979dc426cba0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:08.023984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf4246-3d57-4af0-8520-979dc426cba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:08.083382 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ebbf4246-3d57-4af0-8520-979dc426cba0 None None] MFA Rules not processed for user `8345647e62b64a3e9fe27374209b5722`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:08.086576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ebbf4246-3d57-4af0-8520-979dc426cba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:08.152568 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a4af7-fff1-4f05-9cc1-31e5c8a037d1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:08.178994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:08.209730 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 922/1849] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:08 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:08.210933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf4246-3d57-4af0-8520-979dc426cba0 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:08.214130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ebbf4246-3d57-4af0-8520-979dc426cba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:08.271742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e53170-2b74-4825-98b5-0b0cbfde0ed6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:08.272757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e53170-2b74-4825-98b5-0b0cbfde0ed6 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:08.273039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e53170-2b74-4825-98b5-0b0cbfde0ed6 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:08.273314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e53170-2b74-4825-98b5-0b0cbfde0ed6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:08.283348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e53170-2b74-4825-98b5-0b0cbfde0ed6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:08.285137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e53170-2b74-4825-98b5-0b0cbfde0ed6 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:08.296956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 928/1850] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:08.297644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e53170-2b74-4825-98b5-0b0cbfde0ed6 service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:08.320274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:08.324095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf4246-3d57-4af0-8520-979dc426cba0 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:08.331202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:08.332246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:08.332587 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:08.333015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] PATH_INFO: `/v3/projects/eda7dce32b864d139d83f66f7b4fed2f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:08.335665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eda7dce32b864d139d83f66f7b4fed2f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:08.338051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:08.365188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e53170-2b74-4825-98b5-0b0cbfde0ed6 service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:08.370343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d8d10-d0db-4aba-89a9-85a1c8c91960 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:08.371016 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d8d10-d0db-4aba-89a9-85a1c8c91960 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:08.371304 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d8d10-d0db-4aba-89a9-85a1c8c91960 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:08.371586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d8d10-d0db-4aba-89a9-85a1c8c91960 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:08.383519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d8d10-d0db-4aba-89a9-85a1c8c91960 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:08.384989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d8d10-d0db-4aba-89a9-85a1c8c91960 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:08.395976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 929/1851] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:08.400286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eda7dce32b864d139d83f66f7b4fed2f', 'request_id': 'req-27b191ad-e691-46f5-820b-7bae58f957d9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:08.406292 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eda7dce32b864d139d83f66f7b4fed2f', 'request_id': 'req-27b191ad-e691-46f5-820b-7bae58f957d9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:08.413797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eda7dce32b864d139d83f66f7b4fed2f', 'request_id': 'req-27b191ad-e691-46f5-820b-7bae58f957d9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:08.429965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] The token cache is being invalidate because project eda7dce32b864d139d83f66f7b4fed2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:08.430188 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:08.434074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:08.435747 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 923/1852] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:08 2022] DELETE /identity/v3/projects/eda7dce32b864d139d83f66f7b4fed2f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:08.439609 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d8d10-d0db-4aba-89a9-85a1c8c91960 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:08.450305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d8d10-d0db-4aba-89a9-85a1c8c91960 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:08.479269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:08.480766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:08.511260 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:08.512169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:08.512720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:08.512841 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] PATH_INFO: `/v3/users/85a63cad2df546abac0bca6f94d4f305` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:08.514075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85a63cad2df546abac0bca6f94d4f305)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:08.516324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:08.543118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b191ad-e691-46f5-820b-7bae58f957d9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:08.547135 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92133bc7-1347-46dd-bd1f-009997a9e76d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:08.547814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92133bc7-1347-46dd-bd1f-009997a9e76d service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:08.548049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92133bc7-1347-46dd-bd1f-009997a9e76d service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:08.548358 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92133bc7-1347-46dd-bd1f-009997a9e76d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:08.550355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-92133bc7-1347-46dd-bd1f-009997a9e76d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:08.578076 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85a63cad2df546abac0bca6f94d4f305'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:08.578789 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85a63cad2df546abac0bca6f94d4f305'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:08.588462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85a63cad2df546abac0bca6f94d4f305'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:08.590387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:08.605896 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85a63cad2df546abac0bca6f94d4f305'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:08.607757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85a63cad2df546abac0bca6f94d4f305'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:08.629612 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 930/1853] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:08 2022] DELETE /identity/v3/users/85a63cad2df546abac0bca6f94d4f305 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:08.635356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92133bc7-1347-46dd-bd1f-009997a9e76d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:08.638461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92133bc7-1347-46dd-bd1f-009997a9e76d service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:08.651185 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 924/1854] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:09.797773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:09.799790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:09.851916 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:09.874783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6516bb1-bf9b-4979-b761-4c7e61a6d1d1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:09.879649 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725b764-6f97-495a-8262-e1f53057292b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:09.880535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725b764-6f97-495a-8262-e1f53057292b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:09.880821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725b764-6f97-495a-8262-e1f53057292b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:09.881101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725b764-6f97-495a-8262-e1f53057292b admin admin] PATH_INFO: `/v3/projects/170dad11fe8f4dc78ae64fc2cd79bf88` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:09.882325 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725b764-6f97-495a-8262-e1f53057292b admin admin] RBAC: Authorizing `identity:delete_project(project_id=170dad11fe8f4dc78ae64fc2cd79bf88)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:09.884261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725b764-6f97-495a-8262-e1f53057292b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:09.938127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2725b764-6f97-495a-8262-e1f53057292b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '170dad11fe8f4dc78ae64fc2cd79bf88', 'request_id': 'req-2725b764-6f97-495a-8262-e1f53057292b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:09.945167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2725b764-6f97-495a-8262-e1f53057292b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '170dad11fe8f4dc78ae64fc2cd79bf88', 'request_id': 'req-2725b764-6f97-495a-8262-e1f53057292b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:09.951167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2725b764-6f97-495a-8262-e1f53057292b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '170dad11fe8f4dc78ae64fc2cd79bf88', 'request_id': 'req-2725b764-6f97-495a-8262-e1f53057292b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:09.964841 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2725b764-6f97-495a-8262-e1f53057292b admin admin] The token cache is being invalidate because project 170dad11fe8f4dc78ae64fc2cd79bf88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:09.965138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2725b764-6f97-495a-8262-e1f53057292b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:09.965812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2725b764-6f97-495a-8262-e1f53057292b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:09.967392 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 931/1855] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:09 2022] DELETE /identity/v3/projects/170dad11fe8f4dc78ae64fc2cd79bf88 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:09.972934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918ee6b-233b-41e9-b1cc-8b7a1e0e91c6 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:09.973450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918ee6b-233b-41e9-b1cc-8b7a1e0e91c6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:09.973548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918ee6b-233b-41e9-b1cc-8b7a1e0e91c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:09.991513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2918ee6b-233b-41e9-b1cc-8b7a1e0e91c6 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:09.992726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2918ee6b-233b-41e9-b1cc-8b7a1e0e91c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:10.056087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 925/1856] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:10.063787 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8784040-fe4b-4fbc-ba5f-00456f39f22d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.064144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8784040-fe4b-4fbc-ba5f-00456f39f22d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.064607 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8784040-fe4b-4fbc-ba5f-00456f39f22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.083116 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c8784040-fe4b-4fbc-ba5f-00456f39f22d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:10.084503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c8784040-fe4b-4fbc-ba5f-00456f39f22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:10.096040 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 932/1857] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:10.097414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918ee6b-233b-41e9-b1cc-8b7a1e0e91c6 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:10.114046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918ee6b-233b-41e9-b1cc-8b7a1e0e91c6 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:10.117991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085401d-897e-4c07-9311-d3610f74c630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:10.118681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085401d-897e-4c07-9311-d3610f74c630 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.118963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085401d-897e-4c07-9311-d3610f74c630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.119238 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085401d-897e-4c07-9311-d3610f74c630 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.119976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085401d-897e-4c07-9311-d3610f74c630 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:09:10.120286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085401d-897e-4c07-9311-d3610f74c630 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:10.126754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085401d-897e-4c07-9311-d3610f74c630 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:10.135501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 926/1858] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:09:10.139879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8784040-fe4b-4fbc-ba5f-00456f39f22d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:10.149308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8784040-fe4b-4fbc-ba5f-00456f39f22d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:10.154350 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602c7f5-3d02-4b19-8a71-d1fda236bf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:10.155101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602c7f5-3d02-4b19-8a71-d1fda236bf21 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.155300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602c7f5-3d02-4b19-8a71-d1fda236bf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.155574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602c7f5-3d02-4b19-8a71-d1fda236bf21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.157370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602c7f5-3d02-4b19-8a71-d1fda236bf21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:10.159985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602c7f5-3d02-4b19-8a71-d1fda236bf21 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:10.198280 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 933/1859] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:10 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:10.201109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085401d-897e-4c07-9311-d3610f74c630 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:10.214388 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085401d-897e-4c07-9311-d3610f74c630 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:10.279641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047df5d-0eeb-4235-a5df-dadd1da12f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:10.280333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047df5d-0eeb-4235-a5df-dadd1da12f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.280613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047df5d-0eeb-4235-a5df-dadd1da12f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.280890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047df5d-0eeb-4235-a5df-dadd1da12f48 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.281576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047df5d-0eeb-4235-a5df-dadd1da12f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:10.283650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047df5d-0eeb-4235-a5df-dadd1da12f48 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:10.309666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 927/1860] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:10 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:09:10.312990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602c7f5-3d02-4b19-8a71-d1fda236bf21 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:10.324453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602c7f5-3d02-4b19-8a71-d1fda236bf21 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:10.329257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5e434-d056-4a4a-8559-753fe19d7ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:10.330019 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5e434-d056-4a4a-8559-753fe19d7ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.330662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5e434-d056-4a4a-8559-753fe19d7ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.330949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5e434-d056-4a4a-8559-753fe19d7ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.331699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5e434-d056-4a4a-8559-753fe19d7ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:10.333341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5e434-d056-4a4a-8559-753fe19d7ccc admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:10.348334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 934/1861] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:10.352460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047df5d-0eeb-4235-a5df-dadd1da12f48 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:10.365531 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047df5d-0eeb-4235-a5df-dadd1da12f48 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:10.370620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88643c-a5ff-452c-9ff0-40ad3cd2cdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:10.371304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88643c-a5ff-452c-9ff0-40ad3cd2cdd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.371588 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88643c-a5ff-452c-9ff0-40ad3cd2cdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.371872 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88643c-a5ff-452c-9ff0-40ad3cd2cdd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.372599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88643c-a5ff-452c-9ff0-40ad3cd2cdd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:10.374293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88643c-a5ff-452c-9ff0-40ad3cd2cdd4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:10.387256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 928/1862] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:10.390780 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5e434-d056-4a4a-8559-753fe19d7ccc admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:10.403696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5e434-d056-4a4a-8559-753fe19d7ccc admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:10.409602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f728e7-fe92-4cf4-b23e-ee794dbb88d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:10.410667 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f728e7-fe92-4cf4-b23e-ee794dbb88d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.411015 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f728e7-fe92-4cf4-b23e-ee794dbb88d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.411364 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f728e7-fe92-4cf4-b23e-ee794dbb88d9 admin admin] PATH_INFO: `/v3/projects/363f24ac0a1b4c6b8019307984688897/users/e7618e91240c4ce5ac3ea422479165c5/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.427926 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f728e7-fe92-4cf4-b23e-ee794dbb88d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=363f24ac0a1b4c6b8019307984688897, user_id=e7618e91240c4ce5ac3ea422479165c5, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:10.431615 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f728e7-fe92-4cf4-b23e-ee794dbb88d9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:10.447703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 935/1863] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:10 2022] PUT /identity/v3/projects/363f24ac0a1b4c6b8019307984688897/users/e7618e91240c4ce5ac3ea422479165c5/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:09:10.458096 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751051a-0f7e-418c-95e6-04689dae040e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.458389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751051a-0f7e-418c-95e6-04689dae040e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.458667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751051a-0f7e-418c-95e6-04689dae040e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.486690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7751051a-0f7e-418c-95e6-04689dae040e None None] MFA Rules not processed for user `e7618e91240c4ce5ac3ea422479165c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:10.487859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7751051a-0f7e-418c-95e6-04689dae040e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:10.518571 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:10.555376 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 929/1864] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:10.559964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f728e7-fe92-4cf4-b23e-ee794dbb88d9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:10.628999 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f728e7-fe92-4cf4-b23e-ee794dbb88d9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:10.633943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85705c6-a693-45a4-b97f-4eff40a445e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:10.634654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85705c6-a693-45a4-b97f-4eff40a445e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.634946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85705c6-a693-45a4-b97f-4eff40a445e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.635217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85705c6-a693-45a4-b97f-4eff40a445e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.635915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85705c6-a693-45a4-b97f-4eff40a445e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:10.637886 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85705c6-a693-45a4-b97f-4eff40a445e7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:10.651692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:10.671575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 936/1865] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:10 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:09:10.676648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751051a-0f7e-418c-95e6-04689dae040e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:10.687927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751051a-0f7e-418c-95e6-04689dae040e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:10.747537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48039ba4-50de-49af-bebc-8bc508f5e800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:10.748133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48039ba4-50de-49af-bebc-8bc508f5e800 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.748425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48039ba4-50de-49af-bebc-8bc508f5e800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.748702 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48039ba4-50de-49af-bebc-8bc508f5e800 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.749391 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48039ba4-50de-49af-bebc-8bc508f5e800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:10.751460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48039ba4-50de-49af-bebc-8bc508f5e800 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:10.774054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 930/1866] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:10 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:10.778008 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85705c6-a693-45a4-b97f-4eff40a445e7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:10.788681 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85705c6-a693-45a4-b97f-4eff40a445e7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:10.794236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133675f-9b8e-41ff-968d-4859a2d0aa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:10.794914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133675f-9b8e-41ff-968d-4859a2d0aa79 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.795254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133675f-9b8e-41ff-968d-4859a2d0aa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.795522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133675f-9b8e-41ff-968d-4859a2d0aa79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.796441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133675f-9b8e-41ff-968d-4859a2d0aa79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:10.797871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133675f-9b8e-41ff-968d-4859a2d0aa79 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:10.809363 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 937/1867] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:10.813601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48039ba4-50de-49af-bebc-8bc508f5e800 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:10.824387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48039ba4-50de-49af-bebc-8bc508f5e800 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:10.829167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac29ce1-b182-44f0-8b81-4068df1d248b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:10.829874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac29ce1-b182-44f0-8b81-4068df1d248b admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.830189 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac29ce1-b182-44f0-8b81-4068df1d248b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.830472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac29ce1-b182-44f0-8b81-4068df1d248b admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.831201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac29ce1-b182-44f0-8b81-4068df1d248b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:10.832813 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac29ce1-b182-44f0-8b81-4068df1d248b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:10.847142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 931/1868] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:09:10.851274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133675f-9b8e-41ff-968d-4859a2d0aa79 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:10.861347 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133675f-9b8e-41ff-968d-4859a2d0aa79 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:10.866000 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d94cf-a560-4320-8b9e-732fe6147a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:10.866902 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d94cf-a560-4320-8b9e-732fe6147a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.867182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d94cf-a560-4320-8b9e-732fe6147a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.867459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d94cf-a560-4320-8b9e-732fe6147a38 admin admin] PATH_INFO: `/v3/projects/64eb2a8a7c7c4bfbb6833500059b638d/users/5f35c2a1654249cd92d0b44f86f073df/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.883374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d94cf-a560-4320-8b9e-732fe6147a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64eb2a8a7c7c4bfbb6833500059b638d, user_id=5f35c2a1654249cd92d0b44f86f073df, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:10.886088 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d94cf-a560-4320-8b9e-732fe6147a38 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:10.900610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 938/1869] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:10 2022] PUT /identity/v3/projects/64eb2a8a7c7c4bfbb6833500059b638d/users/5f35c2a1654249cd92d0b44f86f073df/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:10.913282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e3bdb-3a76-4cc4-81a1-ad71884f64ea None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:10.913612 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e3bdb-3a76-4cc4-81a1-ad71884f64ea None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:10.913919 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e3bdb-3a76-4cc4-81a1-ad71884f64ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:10.943780 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-3d9e3bdb-3a76-4cc4-81a1-ad71884f64ea None None] MFA Rules not processed for user `5f35c2a1654249cd92d0b44f86f073df`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:10.945028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3d9e3bdb-3a76-4cc4-81a1-ad71884f64ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:11.022768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 932/1870] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:11.027193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d94cf-a560-4320-8b9e-732fe6147a38 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:11.090827 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d94cf-a560-4320-8b9e-732fe6147a38 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:11.096699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9e56f-351a-4149-8c83-b9bb54072689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:11.097394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9e56f-351a-4149-8c83-b9bb54072689 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:11.097714 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9e56f-351a-4149-8c83-b9bb54072689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:11.097991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9e56f-351a-4149-8c83-b9bb54072689 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:11.098691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9e56f-351a-4149-8c83-b9bb54072689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:11.100668 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9e56f-351a-4149-8c83-b9bb54072689 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:11.136812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 939/1871] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:11 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:11.140473 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e3bdb-3a76-4cc4-81a1-ad71884f64ea None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:11.152196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e3bdb-3a76-4cc4-81a1-ad71884f64ea None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:11.189980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:11.212070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229a8c5-8441-4a49-867d-dc0bc5056b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:11.212895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229a8c5-8441-4a49-867d-dc0bc5056b40 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:11.213271 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229a8c5-8441-4a49-867d-dc0bc5056b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:11.213688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229a8c5-8441-4a49-867d-dc0bc5056b40 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:11.215700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229a8c5-8441-4a49-867d-dc0bc5056b40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:11.222444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229a8c5-8441-4a49-867d-dc0bc5056b40 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:11.241673 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 933/1872] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:11 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:11.245722 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9e56f-351a-4149-8c83-b9bb54072689 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:11.256565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9e56f-351a-4149-8c83-b9bb54072689 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:11.261525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bc6de-ad03-4318-9ba2-52be402d2f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:11.262303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bc6de-ad03-4318-9ba2-52be402d2f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:11.262585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bc6de-ad03-4318-9ba2-52be402d2f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:11.262860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bc6de-ad03-4318-9ba2-52be402d2f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:11.263621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bc6de-ad03-4318-9ba2-52be402d2f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:11.265223 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bc6de-ad03-4318-9ba2-52be402d2f99 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:11.281690 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 940/1873] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:11.282910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229a8c5-8441-4a49-867d-dc0bc5056b40 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:11.296469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229a8c5-8441-4a49-867d-dc0bc5056b40 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:11.302087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2614f15-67b5-454d-b097-d573ae77c3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:11.303013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2614f15-67b5-454d-b097-d573ae77c3f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:11.303295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2614f15-67b5-454d-b097-d573ae77c3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:11.303596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2614f15-67b5-454d-b097-d573ae77c3f5 admin admin] PATH_INFO: `/v3/projects/9f53034543864de4a8b2a8e22be51996/users/ce8de753a6074503af7ad7db7d8875a4/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:11.319756 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2614f15-67b5-454d-b097-d573ae77c3f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f53034543864de4a8b2a8e22be51996, user_id=ce8de753a6074503af7ad7db7d8875a4, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:11.322505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2614f15-67b5-454d-b097-d573ae77c3f5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:11.336966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 934/1874] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:11 2022] PUT /identity/v3/projects/9f53034543864de4a8b2a8e22be51996/users/ce8de753a6074503af7ad7db7d8875a4/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:11.341102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bc6de-ad03-4318-9ba2-52be402d2f99 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:11.352236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bc6de-ad03-4318-9ba2-52be402d2f99 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:11.408506 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33801347-33e6-490f-8b0d-4fc16fa5926e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:11.409243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33801347-33e6-490f-8b0d-4fc16fa5926e admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:11.409522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33801347-33e6-490f-8b0d-4fc16fa5926e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:11.409829 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33801347-33e6-490f-8b0d-4fc16fa5926e admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:11.410550 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33801347-33e6-490f-8b0d-4fc16fa5926e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:11.412149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33801347-33e6-490f-8b0d-4fc16fa5926e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:11.421758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 941/1875] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:11.426469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2614f15-67b5-454d-b097-d573ae77c3f5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:11.437817 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2614f15-67b5-454d-b097-d573ae77c3f5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:11.443153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b03e7-3997-4010-81a1-d4159bfa5858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:11.444057 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b03e7-3997-4010-81a1-d4159bfa5858 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:11.444342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b03e7-3997-4010-81a1-d4159bfa5858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:11.444623 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b03e7-3997-4010-81a1-d4159bfa5858 admin admin] PATH_INFO: `/v3/projects/9f53034543864de4a8b2a8e22be51996/users/ce8de753a6074503af7ad7db7d8875a4/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:11.446735 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b03e7-3997-4010-81a1-d4159bfa5858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f53034543864de4a8b2a8e22be51996, user_id=ce8de753a6074503af7ad7db7d8875a4, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:11.449404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b03e7-3997-4010-81a1-d4159bfa5858 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:11.460167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 935/1876] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:11 2022] PUT /identity/v3/projects/9f53034543864de4a8b2a8e22be51996/users/ce8de753a6074503af7ad7db7d8875a4/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:11.470752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7471ff1-e412-41ff-9de8-8d4febf87e97 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:11.471107 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7471ff1-e412-41ff-9de8-8d4febf87e97 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:11.471525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7471ff1-e412-41ff-9de8-8d4febf87e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:11.500429 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f7471ff1-e412-41ff-9de8-8d4febf87e97 None None] MFA Rules not processed for user `ce8de753a6074503af7ad7db7d8875a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:11.501721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f7471ff1-e412-41ff-9de8-8d4febf87e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:11.520557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:11.576050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 942/1877] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:11.592731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b03e7-3997-4010-81a1-d4159bfa5858 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:11.596075 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f49b03e7-3997-4010-81a1-d4159bfa5858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:11.653418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b03e7-3997-4010-81a1-d4159bfa5858 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:11.657449 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde6c91-8e3e-416f-8035-45d443684f40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:11.658119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde6c91-8e3e-416f-8035-45d443684f40 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:11.658457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde6c91-8e3e-416f-8035-45d443684f40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:11.658729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde6c91-8e3e-416f-8035-45d443684f40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:11.669149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde6c91-8e3e-416f-8035-45d443684f40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:11.670643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde6c91-8e3e-416f-8035-45d443684f40 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:11.680926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 936/1878] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:11.723392 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7471ff1-e412-41ff-9de8-8d4febf87e97 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:11.725134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f7471ff1-e412-41ff-9de8-8d4febf87e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:11.738831 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7471ff1-e412-41ff-9de8-8d4febf87e97 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:11.743356 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058fd0d-f76b-4fc9-b579-a29f84d1d0a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:11.744220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058fd0d-f76b-4fc9-b579-a29f84d1d0a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:11.744371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058fd0d-f76b-4fc9-b579-a29f84d1d0a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:11.744613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058fd0d-f76b-4fc9-b579-a29f84d1d0a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:11.837233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058fd0d-f76b-4fc9-b579-a29f84d1d0a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:11.838220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058fd0d-f76b-4fc9-b579-a29f84d1d0a5 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:11.859417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 943/1879] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:14.663831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa5732-856d-45d2-a357-4c398014ef23 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:14.664356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa5732-856d-45d2-a357-4c398014ef23 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:14.665079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa5732-856d-45d2-a357-4c398014ef23 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:14.667753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 937/1880] 10.222.0.79 () {54 vars in 921 bytes} [Mon Jul 4 10:09:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:09:14.703913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaca54-5c7b-4290-b3dc-6d453a48f6cb None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:14.704513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaca54-5c7b-4290-b3dc-6d453a48f6cb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:14.704513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaca54-5c7b-4290-b3dc-6d453a48f6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:14.742845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3dcaca54-5c7b-4290-b3dc-6d453a48f6cb None None] MFA Rules not processed for user `ffc2409cb7fc4373a74ddfb48d501d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:14.749009 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3dcaca54-5c7b-4290-b3dc-6d453a48f6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:14.977353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 944/1881] 10.222.0.79 () {58 vars in 1018 bytes} [Mon Jul 4 10:09:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:09:15.711091 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa5732-856d-45d2-a357-4c398014ef23 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:15.752123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaca54-5c7b-4290-b3dc-6d453a48f6cb None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:15.754027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3dcaca54-5c7b-4290-b3dc-6d453a48f6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:15.775501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaca54-5c7b-4290-b3dc-6d453a48f6cb None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:15.791904 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dae28e-9df1-48e6-8080-20fd2c90e2fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:15.792655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae28e-9df1-48e6-8080-20fd2c90e2fd service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:15.793037 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae28e-9df1-48e6-8080-20fd2c90e2fd service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:15.794034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae28e-9df1-48e6-8080-20fd2c90e2fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:15.811899 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dae28e-9df1-48e6-8080-20fd2c90e2fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:15.814167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dae28e-9df1-48e6-8080-20fd2c90e2fd service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:15.825900 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 945/1882] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:09:15.847052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa5732-856d-45d2-a357-4c398014ef23 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:15.854688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573b2df-cb7d-4bd4-82a0-011eb8f42fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:15.856218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b2df-cb7d-4bd4-82a0-011eb8f42fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:15.856495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b2df-cb7d-4bd4-82a0-011eb8f42fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:15.856869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b2df-cb7d-4bd4-82a0-011eb8f42fc2 admin admin] PATH_INFO: `/v3/users/e7618e91240c4ce5ac3ea422479165c5` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:15.858231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573b2df-cb7d-4bd4-82a0-011eb8f42fc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7618e91240c4ce5ac3ea422479165c5)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:15.860044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573b2df-cb7d-4bd4-82a0-011eb8f42fc2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:15.918438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:15.933584 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4573b2df-cb7d-4bd4-82a0-011eb8f42fc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7618e91240c4ce5ac3ea422479165c5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:15.934450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4573b2df-cb7d-4bd4-82a0-011eb8f42fc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7618e91240c4ce5ac3ea422479165c5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:15.942584 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4573b2df-cb7d-4bd4-82a0-011eb8f42fc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7618e91240c4ce5ac3ea422479165c5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:15.956245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4573b2df-cb7d-4bd4-82a0-011eb8f42fc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7618e91240c4ce5ac3ea422479165c5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:15.957208 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4573b2df-cb7d-4bd4-82a0-011eb8f42fc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7618e91240c4ce5ac3ea422479165c5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:15.962800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dae28e-9df1-48e6-8080-20fd2c90e2fd service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:15.964310 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f3dae28e-9df1-48e6-8080-20fd2c90e2fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:15.972229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 938/1883] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:15 2022] DELETE /identity/v3/users/e7618e91240c4ce5ac3ea422479165c5 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:15.975366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f552414-c3d1-473b-ab02-11546376a48e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:15.975883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f552414-c3d1-473b-ab02-11546376a48e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:15.976256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f552414-c3d1-473b-ab02-11546376a48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:15.980341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dae28e-9df1-48e6-8080-20fd2c90e2fd service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:15.995801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4f552414-c3d1-473b-ab02-11546376a48e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:15.997030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4f552414-c3d1-473b-ab02-11546376a48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:16.018243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:16.058683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:16.060158 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:16.060679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:16.061099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] PATH_INFO: `/v3/users/b6437457fb0a4512bcdd8994cfa6af75` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:16.062154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 939/1884] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:09:16.062933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6437457fb0a4512bcdd8994cfa6af75)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:16.066288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:16.073058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f552414-c3d1-473b-ab02-11546376a48e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:16.075053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4f552414-c3d1-473b-ab02-11546376a48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:16.123542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6437457fb0a4512bcdd8994cfa6af75'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.124242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6437457fb0a4512bcdd8994cfa6af75'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.135417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6437457fb0a4512bcdd8994cfa6af75'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.138656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f552414-c3d1-473b-ab02-11546376a48e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:16.143095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50d68d-c024-4f78-83d5-b485ab53ca20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:16.143753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50d68d-c024-4f78-83d5-b485ab53ca20 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:16.144031 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50d68d-c024-4f78-83d5-b485ab53ca20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:16.144305 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50d68d-c024-4f78-83d5-b485ab53ca20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:16.149713 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6437457fb0a4512bcdd8994cfa6af75'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.150468 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6437457fb0a4512bcdd8994cfa6af75'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.154300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50d68d-c024-4f78-83d5-b485ab53ca20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:16.156072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50d68d-c024-4f78-83d5-b485ab53ca20 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:16.163273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 946/1885] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:15 2022] DELETE /identity/v3/users/b6437457fb0a4512bcdd8994cfa6af75 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:09:16.165524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 940/1886] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:09:16.167582 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:16.167844 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b417af8-2dcd-4acd-a2a8-1b23adae78c3 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:16.168249 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b417af8-2dcd-4acd-a2a8-1b23adae78c3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:16.168550 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b417af8-2dcd-4acd-a2a8-1b23adae78c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:16.169044 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:16.186518 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6b417af8-2dcd-4acd-a2a8-1b23adae78c3 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:16.187804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6b417af8-2dcd-4acd-a2a8-1b23adae78c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:16.243771 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550ba1b-08d4-43e4-9482-5ba69767efcb admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:16.249114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cf9d3-6058-454a-91e1-9755113ac31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:16.250183 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cf9d3-6058-454a-91e1-9755113ac31e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:16.250519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cf9d3-6058-454a-91e1-9755113ac31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:16.250799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cf9d3-6058-454a-91e1-9755113ac31e admin admin] PATH_INFO: `/v3/users/291be646e79641dfa03cf3eb41d39bad` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:16.251484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:16.253170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cf9d3-6058-454a-91e1-9755113ac31e admin admin] RBAC: Authorizing `identity:delete_user(user_id=291be646e79641dfa03cf3eb41d39bad)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:16.255366 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cf9d3-6058-454a-91e1-9755113ac31e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:16.283685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 941/1887] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:09:16.305829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b417af8-2dcd-4acd-a2a8-1b23adae78c3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:16.307688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6b417af8-2dcd-4acd-a2a8-1b23adae78c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:16.371130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-405cf9d3-6058-454a-91e1-9755113ac31e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291be646e79641dfa03cf3eb41d39bad'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.372171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-405cf9d3-6058-454a-91e1-9755113ac31e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291be646e79641dfa03cf3eb41d39bad'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.383959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:16.393413 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-405cf9d3-6058-454a-91e1-9755113ac31e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291be646e79641dfa03cf3eb41d39bad'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.412217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b417af8-2dcd-4acd-a2a8-1b23adae78c3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:16.419248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb74504-b8c8-4ffc-8b2e-11841b099611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:16.420888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb74504-b8c8-4ffc-8b2e-11841b099611 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:16.421495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb74504-b8c8-4ffc-8b2e-11841b099611 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:16.422171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb74504-b8c8-4ffc-8b2e-11841b099611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:16.428796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-405cf9d3-6058-454a-91e1-9755113ac31e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291be646e79641dfa03cf3eb41d39bad'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.429802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-405cf9d3-6058-454a-91e1-9755113ac31e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291be646e79641dfa03cf3eb41d39bad'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.442847 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb74504-b8c8-4ffc-8b2e-11841b099611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:16.445889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb74504-b8c8-4ffc-8b2e-11841b099611 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:16.464833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 947/1888] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:16 2022] DELETE /identity/v3/users/291be646e79641dfa03cf3eb41d39bad => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:16.483393 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9f251-581e-4960-adc9-20bfe826ce48 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:16.484896 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9f251-581e-4960-adc9-20bfe826ce48 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:16.485568 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9f251-581e-4960-adc9-20bfe826ce48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:16.500462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 942/1889] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:09:16.503192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb74504-b8c8-4ffc-8b2e-11841b099611 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:16.507245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9cb74504-b8c8-4ffc-8b2e-11841b099611 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:16.528709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7be9f251-581e-4960-adc9-20bfe826ce48 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:16.532076 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7be9f251-581e-4960-adc9-20bfe826ce48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:16.532255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb74504-b8c8-4ffc-8b2e-11841b099611 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:16.631639 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:16.632959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:16.632959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:16.633281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] PATH_INFO: `/v3/projects/363f24ac0a1b4c6b8019307984688897` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:16.634821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=363f24ac0a1b4c6b8019307984688897)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:16.636840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:16.682085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 948/1890] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:16.691782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9f251-581e-4960-adc9-20bfe826ce48 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:16.694270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7be9f251-581e-4960-adc9-20bfe826ce48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:16.733123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '363f24ac0a1b4c6b8019307984688897', 'request_id': 'req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.755874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '363f24ac0a1b4c6b8019307984688897', 'request_id': 'req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.767305 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '363f24ac0a1b4c6b8019307984688897', 'request_id': 'req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.790620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] The token cache is being invalidate because project 363f24ac0a1b4c6b8019307984688897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:16.791889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.796382 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:16.800721 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 943/1891] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:16 2022] DELETE /identity/v3/projects/363f24ac0a1b4c6b8019307984688897 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:16.804918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:16.806710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:16.823823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9f251-581e-4960-adc9-20bfe826ce48 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:16.883515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5cabd-1d3e-48b4-a7d3-ecc4576ff9ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:16.884191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5cabd-1d3e-48b4-a7d3-ecc4576ff9ca service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:16.884477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5cabd-1d3e-48b4-a7d3-ecc4576ff9ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:16.884759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5cabd-1d3e-48b4-a7d3-ecc4576ff9ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:16.887104 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b5d5cabd-1d3e-48b4-a7d3-ecc4576ff9ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:16.924192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:16.949051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:16.975357 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5cabd-1d3e-48b4-a7d3-ecc4576ff9ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:16.977028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5cabd-1d3e-48b4-a7d3-ecc4576ff9ca service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:16.978702 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc4bbd-a9bb-4563-830d-84f2f914c4f3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:16.987477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:16.988402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:16.988879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:16.989164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] PATH_INFO: `/v3/users/5f35c2a1654249cd92d0b44f86f073df` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:16.991615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f35c2a1654249cd92d0b44f86f073df)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:16.993234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 949/1892] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:16.999281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:17.086605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5cabd-1d3e-48b4-a7d3-ecc4576ff9ca service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:17.088274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b5d5cabd-1d3e-48b4-a7d3-ecc4576ff9ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:17.106572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f35c2a1654249cd92d0b44f86f073df'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.106572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f35c2a1654249cd92d0b44f86f073df'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.109789 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5cabd-1d3e-48b4-a7d3-ecc4576ff9ca service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:17.128191 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f35c2a1654249cd92d0b44f86f073df'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.128308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:17.128957 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:17.129365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:17.134268 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] PATH_INFO: `/v3/projects/d92112fc41964271b6f085fdb7df7f99` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:17.135686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d92112fc41964271b6f085fdb7df7f99)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:17.139260 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:17.147172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f35c2a1654249cd92d0b44f86f073df'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.147941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f35c2a1654249cd92d0b44f86f073df'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.175032 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 944/1893] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:16 2022] DELETE /identity/v3/users/5f35c2a1654249cd92d0b44f86f073df => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:17.230418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd92112fc41964271b6f085fdb7df7f99', 'request_id': 'req-34e6a193-b799-4761-bce2-16a6b74d30e6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.238778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd92112fc41964271b6f085fdb7df7f99', 'request_id': 'req-34e6a193-b799-4761-bce2-16a6b74d30e6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.251326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd92112fc41964271b6f085fdb7df7f99', 'request_id': 'req-34e6a193-b799-4761-bce2-16a6b74d30e6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.278163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] The token cache is being invalidate because project d92112fc41964271b6f085fdb7df7f99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:17.278687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.279544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.282893 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 950/1894] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:17 2022] DELETE /identity/v3/projects/d92112fc41964271b6f085fdb7df7f99 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:09:17.287432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:17.291650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:17.411453 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4471fd-38dd-43b2-96d2-6537d81daad1 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:17.470399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:17.482229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:17.482229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:17.482229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:17.482229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] PATH_INFO: `/v3/users/ca03cf56b1744229941ade383eddef24` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:17.484647 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca03cf56b1744229941ade383eddef24)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:17.487997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:17.561896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca03cf56b1744229941ade383eddef24'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.562800 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca03cf56b1744229941ade383eddef24'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.571826 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca03cf56b1744229941ade383eddef24'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.576360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:17.577980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:17.601684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca03cf56b1744229941ade383eddef24'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.602906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca03cf56b1744229941ade383eddef24'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.634549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6a193-b799-4761-bce2-16a6b74d30e6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:17.635165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 945/1895] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:17 2022] DELETE /identity/v3/users/ca03cf56b1744229941ade383eddef24 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:17.659986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:17.662265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:17.758609 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:17.759456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d057f5e-0f92-4347-9f28-09d4973510dd admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:17.759924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:17.760164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:17.760391 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] PATH_INFO: `/v3/projects/7b726e6855e44839a1c1abeee4d7bdc1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:17.761657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b726e6855e44839a1c1abeee4d7bdc1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:17.765313 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:17.765472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:17.766926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:17.767277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:17.767843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] PATH_INFO: `/v3/projects/64eb2a8a7c7c4bfbb6833500059b638d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:17.770833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=64eb2a8a7c7c4bfbb6833500059b638d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:17.773637 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:17.859038 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64eb2a8a7c7c4bfbb6833500059b638d', 'request_id': 'req-c6ee624c-d721-415e-a186-e41a3ccc7acf'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.866909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b726e6855e44839a1c1abeee4d7bdc1', 'request_id': 'req-161e9c98-e467-4dfc-b85a-20d58fa53048'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.872353 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64eb2a8a7c7c4bfbb6833500059b638d', 'request_id': 'req-c6ee624c-d721-415e-a186-e41a3ccc7acf'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.890811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b726e6855e44839a1c1abeee4d7bdc1', 'request_id': 'req-161e9c98-e467-4dfc-b85a-20d58fa53048'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.891405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:17.896847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b726e6855e44839a1c1abeee4d7bdc1', 'request_id': 'req-161e9c98-e467-4dfc-b85a-20d58fa53048'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.909567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64eb2a8a7c7c4bfbb6833500059b638d', 'request_id': 'req-c6ee624c-d721-415e-a186-e41a3ccc7acf'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.932047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] The token cache is being invalidate because project 7b726e6855e44839a1c1abeee4d7bdc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:17.932626 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.934247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.937708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] The token cache is being invalidate because project 64eb2a8a7c7c4bfbb6833500059b638d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:17.938266 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 951/1896] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:17 2022] DELETE /identity/v3/projects/7b726e6855e44839a1c1abeee4d7bdc1 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:09:17.938459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.940000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:17.941338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:17.943184 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:17.948360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 946/1897] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:17 2022] DELETE /identity/v3/projects/64eb2a8a7c7c4bfbb6833500059b638d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:17.951184 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:17.955707 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:17.989834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:18.105146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee624c-d721-415e-a186-e41a3ccc7acf admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:18.109379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e9c98-e467-4dfc-b85a-20d58fa53048 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:18.117225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:18.118359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:18.118713 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:18.119006 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] PATH_INFO: `/v3/users/ce8de753a6074503af7ad7db7d8875a4` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:18.120911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce8de753a6074503af7ad7db7d8875a4)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:18.128072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:18.129794 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:18.130128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:18.130455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] PATH_INFO: `/v3/users/8345647e62b64a3e9fe27374209b5722` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:18.134972 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:18.136380 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8345647e62b64a3e9fe27374209b5722)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:18.138936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:18.249555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce8de753a6074503af7ad7db7d8875a4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:18.250401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce8de753a6074503af7ad7db7d8875a4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:18.268673 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce8de753a6074503af7ad7db7d8875a4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:18.271602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8345647e62b64a3e9fe27374209b5722'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:18.273573 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8345647e62b64a3e9fe27374209b5722'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:18.290969 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce8de753a6074503af7ad7db7d8875a4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:18.291524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8345647e62b64a3e9fe27374209b5722'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:18.292434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce8de753a6074503af7ad7db7d8875a4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:18.314790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8345647e62b64a3e9fe27374209b5722'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:18.317513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8345647e62b64a3e9fe27374209b5722'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:18.334818 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 947/1898] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:17 2022] DELETE /identity/v3/users/ce8de753a6074503af7ad7db7d8875a4 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:18.355795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 952/1899] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:17 2022] DELETE /identity/v3/users/8345647e62b64a3e9fe27374209b5722 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:09:19.030116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.032351 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:19.112000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d085a-6203-4a08-afb0-02c5ff94ebf4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:19.156568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:19.196775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.197631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c6006-5ccf-4755-9250-6b54246ff442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:19.198622 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c6006-5ccf-4755-9250-6b54246ff442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.198971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c6006-5ccf-4755-9250-6b54246ff442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.199371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c6006-5ccf-4755-9250-6b54246ff442 admin admin] PATH_INFO: `/v3/projects/9f53034543864de4a8b2a8e22be51996` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.200846 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c6006-5ccf-4755-9250-6b54246ff442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f53034543864de4a8b2a8e22be51996)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:19.202818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:19.204608 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c6006-5ccf-4755-9250-6b54246ff442 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:19.245585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04119c1d-f1d9-4d24-a165-f1391e15fe7b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:19.257559 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22b3a7-621d-4c85-98a9-716c330fb9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:19.258782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22b3a7-621d-4c85-98a9-716c330fb9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.259620 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22b3a7-621d-4c85-98a9-716c330fb9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.260231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22b3a7-621d-4c85-98a9-716c330fb9b8 admin admin] PATH_INFO: `/v3/projects/99e7eaca5ea249339279ea246ef84f5a` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.262897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22b3a7-621d-4c85-98a9-716c330fb9b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99e7eaca5ea249339279ea246ef84f5a)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:19.267501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22b3a7-621d-4c85-98a9-716c330fb9b8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:19.271769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-888c6006-5ccf-4755-9250-6b54246ff442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f53034543864de4a8b2a8e22be51996', 'request_id': 'req-888c6006-5ccf-4755-9250-6b54246ff442'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.277662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-888c6006-5ccf-4755-9250-6b54246ff442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f53034543864de4a8b2a8e22be51996', 'request_id': 'req-888c6006-5ccf-4755-9250-6b54246ff442'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.286771 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-888c6006-5ccf-4755-9250-6b54246ff442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f53034543864de4a8b2a8e22be51996', 'request_id': 'req-888c6006-5ccf-4755-9250-6b54246ff442'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.321420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-888c6006-5ccf-4755-9250-6b54246ff442 admin admin] The token cache is being invalidate because project 9f53034543864de4a8b2a8e22be51996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:19.321863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-888c6006-5ccf-4755-9250-6b54246ff442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.322729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-888c6006-5ccf-4755-9250-6b54246ff442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.325399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 948/1900] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:19 2022] DELETE /identity/v3/projects/9f53034543864de4a8b2a8e22be51996 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:19.332515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d0bc8-fc98-4bfc-9c6f-3995516fe2e7 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.333118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d0bc8-fc98-4bfc-9c6f-3995516fe2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.333755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d0bc8-fc98-4bfc-9c6f-3995516fe2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.365103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-174d0bc8-fc98-4bfc-9c6f-3995516fe2e7 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:19.367194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-174d0bc8-fc98-4bfc-9c6f-3995516fe2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:19.375012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4e22b3a7-621d-4c85-98a9-716c330fb9b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99e7eaca5ea249339279ea246ef84f5a', 'request_id': 'req-4e22b3a7-621d-4c85-98a9-716c330fb9b8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.384163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4e22b3a7-621d-4c85-98a9-716c330fb9b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99e7eaca5ea249339279ea246ef84f5a', 'request_id': 'req-4e22b3a7-621d-4c85-98a9-716c330fb9b8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.392289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4e22b3a7-621d-4c85-98a9-716c330fb9b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99e7eaca5ea249339279ea246ef84f5a', 'request_id': 'req-4e22b3a7-621d-4c85-98a9-716c330fb9b8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.394410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:19.423686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4e22b3a7-621d-4c85-98a9-716c330fb9b8 admin admin] The token cache is being invalidate because project 99e7eaca5ea249339279ea246ef84f5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:19.423921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4e22b3a7-621d-4c85-98a9-716c330fb9b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.424926 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4e22b3a7-621d-4c85-98a9-716c330fb9b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.432736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 953/1901] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:19 2022] DELETE /identity/v3/projects/99e7eaca5ea249339279ea246ef84f5a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:19.435574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05c593-f47b-42ac-9fb2-54aca49f8f58 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.436082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05c593-f47b-42ac-9fb2-54aca49f8f58 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.436373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05c593-f47b-42ac-9fb2-54aca49f8f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.457683 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 949/1902] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:19.460242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc397c6-a33e-40dd-bcf0-bd7f1037665e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.460628 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc397c6-a33e-40dd-bcf0-bd7f1037665e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.460968 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ea05c593-f47b-42ac-9fb2-54aca49f8f58 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:19.461100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc397c6-a33e-40dd-bcf0-bd7f1037665e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.462299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ea05c593-f47b-42ac-9fb2-54aca49f8f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:19.481777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 954/1903] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:19.483846 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-9fc397c6-a33e-40dd-bcf0-bd7f1037665e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:19.485177 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9fc397c6-a33e-40dd-bcf0-bd7f1037665e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:19.489738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017dd5a-dc63-4d92-b3f3-b83ab38f2bda None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.490324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017dd5a-dc63-4d92-b3f3-b83ab38f2bda None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.490842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017dd5a-dc63-4d92-b3f3-b83ab38f2bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.496486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 950/1904] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:19.500376 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc397c6-a33e-40dd-bcf0-bd7f1037665e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.512521 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc397c6-a33e-40dd-bcf0-bd7f1037665e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:19.517854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4df8f-0781-4255-b49d-8b0695df157d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:19.518930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4df8f-0781-4255-b49d-8b0695df157d admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.519102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4df8f-0781-4255-b49d-8b0695df157d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.519473 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4df8f-0781-4255-b49d-8b0695df157d admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.520411 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4df8f-0781-4255-b49d-8b0695df157d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:09:19.520830 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4df8f-0781-4255-b49d-8b0695df157d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:19.522755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4df8f-0781-4255-b49d-8b0695df157d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:19.522910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3017dd5a-dc63-4d92-b3f3-b83ab38f2bda None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:19.525607 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3017dd5a-dc63-4d92-b3f3-b83ab38f2bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:19.537560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 951/1905] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:09:19.543734 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 955/1906] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:19.543883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4df8f-0781-4255-b49d-8b0695df157d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.545219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017dd5a-dc63-4d92-b3f3-b83ab38f2bda None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.553110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4df8f-0781-4255-b49d-8b0695df157d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:19.557988 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23305ace-a47f-44c2-8751-23be621a0709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:19.558706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23305ace-a47f-44c2-8751-23be621a0709 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.558931 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23305ace-a47f-44c2-8751-23be621a0709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.559210 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23305ace-a47f-44c2-8751-23be621a0709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.559546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017dd5a-dc63-4d92-b3f3-b83ab38f2bda None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:19.559948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23305ace-a47f-44c2-8751-23be621a0709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:19.561920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23305ace-a47f-44c2-8751-23be621a0709 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:19.565012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999ad4e-636c-449a-98c7-f281bb6d6c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:19.566021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999ad4e-636c-449a-98c7-f281bb6d6c67 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.566768 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999ad4e-636c-449a-98c7-f281bb6d6c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.567053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999ad4e-636c-449a-98c7-f281bb6d6c67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.567850 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0999ad4e-636c-449a-98c7-f281bb6d6c67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:09:19.568166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0999ad4e-636c-449a-98c7-f281bb6d6c67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:19.569820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0999ad4e-636c-449a-98c7-f281bb6d6c67 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:19.593033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 956/1908] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:09:19.595153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999ad4e-636c-449a-98c7-f281bb6d6c67 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.595606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 952/1907] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:19 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:19.596468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23305ace-a47f-44c2-8751-23be621a0709 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.609820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999ad4e-636c-449a-98c7-f281bb6d6c67 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:19.628228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:19.676529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23305ace-a47f-44c2-8751-23be621a0709 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:19.685206 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdf7de-3387-4627-990c-1cbd161e684e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:19.685926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdf7de-3387-4627-990c-1cbd161e684e admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.686215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdf7de-3387-4627-990c-1cbd161e684e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.686487 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdf7de-3387-4627-990c-1cbd161e684e admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.687176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdf7de-3387-4627-990c-1cbd161e684e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:19.689121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdf7de-3387-4627-990c-1cbd161e684e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:19.690642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4f369-2cf1-453f-b049-dbc1563979e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:19.692492 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4f369-2cf1-453f-b049-dbc1563979e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.693131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4f369-2cf1-453f-b049-dbc1563979e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.693924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4f369-2cf1-453f-b049-dbc1563979e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.694639 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4f369-2cf1-453f-b049-dbc1563979e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:19.699542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4f369-2cf1-453f-b049-dbc1563979e6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:19.736409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 953/1909] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:19 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:09:19.737749 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdf7de-3387-4627-990c-1cbd161e684e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.743923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 957/1910] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:19 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:19.748411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4f369-2cf1-453f-b049-dbc1563979e6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.751797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdf7de-3387-4627-990c-1cbd161e684e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:19.817924 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2b545-da67-4625-b18c-21d0539100a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:19.818874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2b545-da67-4625-b18c-21d0539100a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.819203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2b545-da67-4625-b18c-21d0539100a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.819543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2b545-da67-4625-b18c-21d0539100a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.820483 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2b545-da67-4625-b18c-21d0539100a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:19.822423 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2b545-da67-4625-b18c-21d0539100a8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:19.830279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4f369-2cf1-453f-b049-dbc1563979e6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:19.833645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 954/1911] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:19.836239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fed736-4ad0-4ddc-a198-89b98d48ec54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:19.837133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fed736-4ad0-4ddc-a198-89b98d48ec54 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.837569 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fed736-4ad0-4ddc-a198-89b98d48ec54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.837957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2b545-da67-4625-b18c-21d0539100a8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.838111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fed736-4ad0-4ddc-a198-89b98d48ec54 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.839034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fed736-4ad0-4ddc-a198-89b98d48ec54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:19.841450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fed736-4ad0-4ddc-a198-89b98d48ec54 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:19.851193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2b545-da67-4625-b18c-21d0539100a8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:19.859087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c2fcb-11ac-4739-8258-b016ad09a316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:19.859881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c2fcb-11ac-4739-8258-b016ad09a316 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.860242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c2fcb-11ac-4739-8258-b016ad09a316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.860691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c2fcb-11ac-4739-8258-b016ad09a316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.861672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c2fcb-11ac-4739-8258-b016ad09a316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:19.863564 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c2fcb-11ac-4739-8258-b016ad09a316 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:19.872586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 958/1912] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:19 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:09:19.872586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fed736-4ad0-4ddc-a198-89b98d48ec54 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.873128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-12fed736-4ad0-4ddc-a198-89b98d48ec54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:19.879701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 955/1913] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:19.879701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c2fcb-11ac-4739-8258-b016ad09a316 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.888653 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fed736-4ad0-4ddc-a198-89b98d48ec54 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:19.888777 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c2fcb-11ac-4739-8258-b016ad09a316 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:19.893492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1c876-2df6-4290-b81b-cbd5a232026e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:19.894696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1c876-2df6-4290-b81b-cbd5a232026e admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.894986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1c876-2df6-4290-b81b-cbd5a232026e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.895447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1c876-2df6-4290-b81b-cbd5a232026e admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.896303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde1c876-2df6-4290-b81b-cbd5a232026e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:19.897747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:19.898403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde1c876-2df6-4290-b81b-cbd5a232026e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:19.898824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.899138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.899479 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e admin admin] PATH_INFO: `/v3/projects/0efaf2a4ac154b5ea2c1de047312d66f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.901251 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0efaf2a4ac154b5ea2c1de047312d66f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:19.903122 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:19.919836 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 956/1914] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:19.919836 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1c876-2df6-4290-b81b-cbd5a232026e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.928615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1c876-2df6-4290-b81b-cbd5a232026e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:19.934044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3bb7f-5a94-4b24-bcb0-75762d31e302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:19.935014 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3bb7f-5a94-4b24-bcb0-75762d31e302 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.935299 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3bb7f-5a94-4b24-bcb0-75762d31e302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.935579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3bb7f-5a94-4b24-bcb0-75762d31e302 admin admin] PATH_INFO: `/v3/projects/294012b920054904b108e4f0e7d12264/users/43e3bda61ebf4d39b65ba471512d137a/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:19.948384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:19.953687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3bb7f-5a94-4b24-bcb0-75762d31e302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=294012b920054904b108e4f0e7d12264, user_id=43e3bda61ebf4d39b65ba471512d137a, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:19.957040 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3bb7f-5a94-4b24-bcb0-75762d31e302 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:19.960261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0efaf2a4ac154b5ea2c1de047312d66f', 'request_id': 'req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.968616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0efaf2a4ac154b5ea2c1de047312d66f', 'request_id': 'req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.975403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 957/1915] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:19 2022] PUT /identity/v3/projects/294012b920054904b108e4f0e7d12264/users/43e3bda61ebf4d39b65ba471512d137a/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:19.975562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0efaf2a4ac154b5ea2c1de047312d66f', 'request_id': 'req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.978468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3bb7f-5a94-4b24-bcb0-75762d31e302 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:19.992342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e admin admin] The token cache is being invalidate because project 0efaf2a4ac154b5ea2c1de047312d66f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:19.992653 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.993331 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6fefedf8-be22-44b7-a7e8-6f40b2cb4e7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:19.995326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 959/1916] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:19 2022] DELETE /identity/v3/projects/0efaf2a4ac154b5ea2c1de047312d66f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:19.996465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490df7d5-3fb8-48e9-a8d3-d79294b6b208 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:19.996748 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490df7d5-3fb8-48e9-a8d3-d79294b6b208 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:19.997018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490df7d5-3fb8-48e9-a8d3-d79294b6b208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.026114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-490df7d5-3fb8-48e9-a8d3-d79294b6b208 None None] MFA Rules not processed for user `43e3bda61ebf4d39b65ba471512d137a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:20.027461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-490df7d5-3fb8-48e9-a8d3-d79294b6b208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:20.052977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3bb7f-5a94-4b24-bcb0-75762d31e302 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:20.057986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e59829-5b81-4abf-8b24-15f2a6c609e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:20.058669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e59829-5b81-4abf-8b24-15f2a6c609e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.058957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e59829-5b81-4abf-8b24-15f2a6c609e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.059286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e59829-5b81-4abf-8b24-15f2a6c609e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.060021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e59829-5b81-4abf-8b24-15f2a6c609e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:20.061694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e59829-5b81-4abf-8b24-15f2a6c609e2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:20.063841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:20.074692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 958/1917] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:20.076329 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68866f5d-ddf2-4499-b315-4cf8f6ad73bd None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.077003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68866f5d-ddf2-4499-b315-4cf8f6ad73bd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.077286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68866f5d-ddf2-4499-b315-4cf8f6ad73bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.096006 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-68866f5d-ddf2-4499-b315-4cf8f6ad73bd None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:20.097218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-68866f5d-ddf2-4499-b315-4cf8f6ad73bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:20.098447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 960/1918] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:20.099344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490df7d5-3fb8-48e9-a8d3-d79294b6b208 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:20.100567 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-490df7d5-3fb8-48e9-a8d3-d79294b6b208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:20.105301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 959/1919] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:20.110642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68866f5d-ddf2-4499-b315-4cf8f6ad73bd None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:20.112722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-68866f5d-ddf2-4499-b315-4cf8f6ad73bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:20.126273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490df7d5-3fb8-48e9-a8d3-d79294b6b208 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:20.127757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45161213-5739-47ec-8a14-7b37bfc25746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:20.128664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45161213-5739-47ec-8a14-7b37bfc25746 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.128949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45161213-5739-47ec-8a14-7b37bfc25746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.129227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45161213-5739-47ec-8a14-7b37bfc25746 admin admin] PATH_INFO: `/v3/projects/0e9d0fba002a490cbbd2759d5ecfbe75/users/73a55e8e2585484c836048838c54c2e6/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.145047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45161213-5739-47ec-8a14-7b37bfc25746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9d0fba002a490cbbd2759d5ecfbe75, user_id=73a55e8e2585484c836048838c54c2e6, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:20.147749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45161213-5739-47ec-8a14-7b37bfc25746 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:20.162141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 961/1920] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:20 2022] PUT /identity/v3/projects/0e9d0fba002a490cbbd2759d5ecfbe75/users/73a55e8e2585484c836048838c54c2e6/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:20.163348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208d31a-b18b-4f75-b4d4-84b92a204566 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.163636 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208d31a-b18b-4f75-b4d4-84b92a204566 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.163912 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208d31a-b18b-4f75-b4d4-84b92a204566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.171436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68866f5d-ddf2-4499-b315-4cf8f6ad73bd None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:20.183721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f208d31a-b18b-4f75-b4d4-84b92a204566 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:20.184854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f208d31a-b18b-4f75-b4d4-84b92a204566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:20.223660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b696970-6128-4600-b310-c132ce819b0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:20.224323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b696970-6128-4600-b310-c132ce819b0b service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.224610 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b696970-6128-4600-b310-c132ce819b0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.224890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b696970-6128-4600-b310-c132ce819b0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.250474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 962/1921] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:20.252029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e1b71-ed10-4624-8411-3dcddbfca33f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.252404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e1b71-ed10-4624-8411-3dcddbfca33f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.252813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e1b71-ed10-4624-8411-3dcddbfca33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.281101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2d4e1b71-ed10-4624-8411-3dcddbfca33f None None] MFA Rules not processed for user `73a55e8e2585484c836048838c54c2e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:20.282377 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2d4e1b71-ed10-4624-8411-3dcddbfca33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:20.288296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b696970-6128-4600-b310-c132ce819b0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:20.289964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b696970-6128-4600-b310-c132ce819b0b service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:20.299851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 960/1922] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:09:20.300635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b696970-6128-4600-b310-c132ce819b0b service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:20.301252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:20.312144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b696970-6128-4600-b310-c132ce819b0b service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:20.319141 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e020c48-c872-4620-94e5-ca0cc7d4708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:20.319971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e020c48-c872-4620-94e5-ca0cc7d4708d admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.320314 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e020c48-c872-4620-94e5-ca0cc7d4708d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.320600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e020c48-c872-4620-94e5-ca0cc7d4708d admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.321498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e020c48-c872-4620-94e5-ca0cc7d4708d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:09:20.321740 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e020c48-c872-4620-94e5-ca0cc7d4708d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:20.323227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e020c48-c872-4620-94e5-ca0cc7d4708d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:20.339368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 961/1923] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:09:20.339368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e020c48-c872-4620-94e5-ca0cc7d4708d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:20.339368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9e020c48-c872-4620-94e5-ca0cc7d4708d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:20.352253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e020c48-c872-4620-94e5-ca0cc7d4708d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:20.356849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:20.357780 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.358062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.358335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] PATH_INFO: `/v3/users/43e3bda61ebf4d39b65ba471512d137a` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.359711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=43e3bda61ebf4d39b65ba471512d137a)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:20.361758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:20.363680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 963/1924] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:20.364757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e1b71-ed10-4624-8411-3dcddbfca33f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:20.382118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e1b71-ed10-4624-8411-3dcddbfca33f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:20.392314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996f8e1-d626-42ac-8324-ef8bf6298562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:20.393607 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996f8e1-d626-42ac-8324-ef8bf6298562 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.394167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996f8e1-d626-42ac-8324-ef8bf6298562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.394675 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996f8e1-d626-42ac-8324-ef8bf6298562 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.395987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996f8e1-d626-42ac-8324-ef8bf6298562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:20.399985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996f8e1-d626-42ac-8324-ef8bf6298562 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:20.433157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43e3bda61ebf4d39b65ba471512d137a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:20.434588 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43e3bda61ebf4d39b65ba471512d137a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:20.436450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:20.445763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43e3bda61ebf4d39b65ba471512d137a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:20.448251 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 964/1925] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:20 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:20.449263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996f8e1-d626-42ac-8324-ef8bf6298562 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:20.453401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f996f8e1-d626-42ac-8324-ef8bf6298562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:20.463733 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43e3bda61ebf4d39b65ba471512d137a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:20.464711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43e3bda61ebf4d39b65ba471512d137a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:20.484091 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 962/1926] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:20 2022] DELETE /identity/v3/users/43e3bda61ebf4d39b65ba471512d137a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:20.484091 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:20.484494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:20.553971 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996f8e1-d626-42ac-8324-ef8bf6298562 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:20.577985 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64eb9d-1b2a-446a-a9bb-2d9549fe7b8d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:20.585261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fd10c-fc47-4c60-bfd0-eb8abc618fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:20.587082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fd10c-fc47-4c60-bfd0-eb8abc618fef admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.587562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fd10c-fc47-4c60-bfd0-eb8abc618fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.588003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fd10c-fc47-4c60-bfd0-eb8abc618fef admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.588990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fd10c-fc47-4c60-bfd0-eb8abc618fef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:20.592481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fd10c-fc47-4c60-bfd0-eb8abc618fef admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:20.635984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 963/1927] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:20 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:20.638746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0ebe2-2d3c-4f14-928b-43eb9347406e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.639378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0ebe2-2d3c-4f14-928b-43eb9347406e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.639718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0ebe2-2d3c-4f14-928b-43eb9347406e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.668856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8a245-4616-42f6-8fe8-1fada4683205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:20.669916 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8a245-4616-42f6-8fe8-1fada4683205 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.670344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8a245-4616-42f6-8fe8-1fada4683205 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.670895 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8a245-4616-42f6-8fe8-1fada4683205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.672293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-71c0ebe2-2d3c-4f14-928b-43eb9347406e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:20.674191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-97c8a245-4616-42f6-8fe8-1fada4683205 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:20.674813 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-71c0ebe2-2d3c-4f14-928b-43eb9347406e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:20.690808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 964/1928] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:20.692076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0ebe2-2d3c-4f14-928b-43eb9347406e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:20.714308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0ebe2-2d3c-4f14-928b-43eb9347406e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:20.726825 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e3cab-3b8b-4d54-927b-dcda6e2cb89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:20.728215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e3cab-3b8b-4d54-927b-dcda6e2cb89f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.728686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e3cab-3b8b-4d54-927b-dcda6e2cb89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.729136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e3cab-3b8b-4d54-927b-dcda6e2cb89f admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.730348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e3cab-3b8b-4d54-927b-dcda6e2cb89f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:20.735427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e3cab-3b8b-4d54-927b-dcda6e2cb89f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:20.754401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 965/1929] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:20.755782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e3cab-3b8b-4d54-927b-dcda6e2cb89f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:20.758306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-782e3cab-3b8b-4d54-927b-dcda6e2cb89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:20.775435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8a245-4616-42f6-8fe8-1fada4683205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:20.776911 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8a245-4616-42f6-8fe8-1fada4683205 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:20.792257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 965/1930] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:20.793446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8a245-4616-42f6-8fe8-1fada4683205 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:20.809931 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8a245-4616-42f6-8fe8-1fada4683205 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:20.816760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3656f0-cc26-4ac3-bd4f-595972b3d407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:20.817454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3656f0-cc26-4ac3-bd4f-595972b3d407 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.817775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3656f0-cc26-4ac3-bd4f-595972b3d407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.818053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3656f0-cc26-4ac3-bd4f-595972b3d407 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.818971 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3656f0-cc26-4ac3-bd4f-595972b3d407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:20.820708 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3656f0-cc26-4ac3-bd4f-595972b3d407 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:20.822463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:20.837762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 966/1931] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:20.841378 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3656f0-cc26-4ac3-bd4f-595972b3d407 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:20.859059 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3656f0-cc26-4ac3-bd4f-595972b3d407 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:20.865660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e7de0-2309-460e-85e9-34f117ff1b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:20.866704 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e7de0-2309-460e-85e9-34f117ff1b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.867092 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e7de0-2309-460e-85e9-34f117ff1b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.867684 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e7de0-2309-460e-85e9-34f117ff1b5a admin admin] PATH_INFO: `/v3/projects/c5e1edbab5ce4b349ae054b107001758/users/94ac1fa8e37b4814b3aa4d9d0b3c8553/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.915632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e3cab-3b8b-4d54-927b-dcda6e2cb89f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:20.915889 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e7de0-2309-460e-85e9-34f117ff1b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e1edbab5ce4b349ae054b107001758, user_id=94ac1fa8e37b4814b3aa4d9d0b3c8553, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:20.924420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859221c-8a8b-4040-b63a-5e40429b5b34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:20.924732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e7de0-2309-460e-85e9-34f117ff1b5a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:20.926524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859221c-8a8b-4040-b63a-5e40429b5b34 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.927223 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859221c-8a8b-4040-b63a-5e40429b5b34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.931088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859221c-8a8b-4040-b63a-5e40429b5b34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.956522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 967/1932] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:20 2022] PUT /identity/v3/projects/c5e1edbab5ce4b349ae054b107001758/users/94ac1fa8e37b4814b3aa4d9d0b3c8553/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:20.957239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859221c-8a8b-4040-b63a-5e40429b5b34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:20.962628 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859221c-8a8b-4040-b63a-5e40429b5b34 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:20.966943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d6530-9a1d-4e8a-b7b3-20f6e5e1f68c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:20.967314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d6530-9a1d-4e8a-b7b3-20f6e5e1f68c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:20.967741 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d6530-9a1d-4e8a-b7b3-20f6e5e1f68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:20.997141 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 966/1933] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:21.030901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e63d6530-9a1d-4e8a-b7b3-20f6e5e1f68c None None] MFA Rules not processed for user `94ac1fa8e37b4814b3aa4d9d0b3c8553`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:21.032597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e63d6530-9a1d-4e8a-b7b3-20f6e5e1f68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:21.207501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 968/1934] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:21.221587 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859221c-8a8b-4040-b63a-5e40429b5b34 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:21.224506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5859221c-8a8b-4040-b63a-5e40429b5b34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:21.249700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859221c-8a8b-4040-b63a-5e40429b5b34 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:21.338649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:21.384645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b06a9-5224-45cd-a7fb-752e5b565723 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:21.386152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b06a9-5224-45cd-a7fb-752e5b565723 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:21.386511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b06a9-5224-45cd-a7fb-752e5b565723 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:21.386797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b06a9-5224-45cd-a7fb-752e5b565723 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:21.398566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b06a9-5224-45cd-a7fb-752e5b565723 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:21.399876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b06a9-5224-45cd-a7fb-752e5b565723 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:21.424478 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 967/1935] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:21.434216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d6530-9a1d-4e8a-b7b3-20f6e5e1f68c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:21.580605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d6530-9a1d-4e8a-b7b3-20f6e5e1f68c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:21.585882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd1eb8-f91f-4ee8-97c6-2519d9f960a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:21.586856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd1eb8-f91f-4ee8-97c6-2519d9f960a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:21.587058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd1eb8-f91f-4ee8-97c6-2519d9f960a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:21.587332 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd1eb8-f91f-4ee8-97c6-2519d9f960a7 admin admin] PATH_INFO: `/v3/users/94ac1fa8e37b4814b3aa4d9d0b3c8553` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:21.588880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd1eb8-f91f-4ee8-97c6-2519d9f960a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ac1fa8e37b4814b3aa4d9d0b3c8553)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:21.593216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd1eb8-f91f-4ee8-97c6-2519d9f960a7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:21.651582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b06a9-5224-45cd-a7fb-752e5b565723 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:21.653987 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-764b06a9-5224-45cd-a7fb-752e5b565723 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:21.675014 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b06a9-5224-45cd-a7fb-752e5b565723 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:21.681187 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55093e31-b44e-4e72-8de1-52a27a412fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:21.682223 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55093e31-b44e-4e72-8de1-52a27a412fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:21.682620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55093e31-b44e-4e72-8de1-52a27a412fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:21.682902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55093e31-b44e-4e72-8de1-52a27a412fbe admin admin] PATH_INFO: `/v3/projects/294012b920054904b108e4f0e7d12264` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:21.684280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55093e31-b44e-4e72-8de1-52a27a412fbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=294012b920054904b108e4f0e7d12264)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:21.686527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55093e31-b44e-4e72-8de1-52a27a412fbe admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:21.704483 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-61cd1eb8-f91f-4ee8-97c6-2519d9f960a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ac1fa8e37b4814b3aa4d9d0b3c8553'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:21.705983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-61cd1eb8-f91f-4ee8-97c6-2519d9f960a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ac1fa8e37b4814b3aa4d9d0b3c8553'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:21.723029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-61cd1eb8-f91f-4ee8-97c6-2519d9f960a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ac1fa8e37b4814b3aa4d9d0b3c8553'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:21.739399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:21.760505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-61cd1eb8-f91f-4ee8-97c6-2519d9f960a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ac1fa8e37b4814b3aa4d9d0b3c8553'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:21.761727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-61cd1eb8-f91f-4ee8-97c6-2519d9f960a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ac1fa8e37b4814b3aa4d9d0b3c8553'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:21.791329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 969/1936] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:21 2022] DELETE /identity/v3/users/94ac1fa8e37b4814b3aa4d9d0b3c8553 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:21.795745 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80881028-3c92-48d9-a003-174c8e05cc22 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:21.796139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80881028-3c92-48d9-a003-174c8e05cc22 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:21.796512 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80881028-3c92-48d9-a003-174c8e05cc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:21.800964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-55093e31-b44e-4e72-8de1-52a27a412fbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '294012b920054904b108e4f0e7d12264', 'request_id': 'req-55093e31-b44e-4e72-8de1-52a27a412fbe'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:21.812231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-55093e31-b44e-4e72-8de1-52a27a412fbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '294012b920054904b108e4f0e7d12264', 'request_id': 'req-55093e31-b44e-4e72-8de1-52a27a412fbe'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:21.823987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-80881028-3c92-48d9-a003-174c8e05cc22 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:21.825401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-80881028-3c92-48d9-a003-174c8e05cc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:21.842230 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-55093e31-b44e-4e72-8de1-52a27a412fbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '294012b920054904b108e4f0e7d12264', 'request_id': 'req-55093e31-b44e-4e72-8de1-52a27a412fbe'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:21.872613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-55093e31-b44e-4e72-8de1-52a27a412fbe admin admin] The token cache is being invalidate because project 294012b920054904b108e4f0e7d12264 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:21.872993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-55093e31-b44e-4e72-8de1-52a27a412fbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:21.875624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-55093e31-b44e-4e72-8de1-52a27a412fbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:21.881716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 968/1937] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:21 2022] DELETE /identity/v3/projects/294012b920054904b108e4f0e7d12264 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:21.885862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764550f1-d4c2-457a-a594-7cf7eb872d1c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:21.887710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764550f1-d4c2-457a-a594-7cf7eb872d1c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:21.887710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764550f1-d4c2-457a-a594-7cf7eb872d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:21.921969 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-764550f1-d4c2-457a-a594-7cf7eb872d1c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:21.923750 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-764550f1-d4c2-457a-a594-7cf7eb872d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:22.017814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 970/1938] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:09:22.030312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80881028-3c92-48d9-a003-174c8e05cc22 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:22.031284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 969/1939] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:22.038099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-80881028-3c92-48d9-a003-174c8e05cc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:22.039116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79d86b-84ef-4777-8ee3-7443995aa057 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:22.039505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79d86b-84ef-4777-8ee3-7443995aa057 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:22.039873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79d86b-84ef-4777-8ee3-7443995aa057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:22.066825 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-fa79d86b-84ef-4777-8ee3-7443995aa057 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:22.068819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fa79d86b-84ef-4777-8ee3-7443995aa057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:22.080365 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 970/1940] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:09:22.086792 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79d86b-84ef-4777-8ee3-7443995aa057 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:22.096317 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79d86b-84ef-4777-8ee3-7443995aa057 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:22.102945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb419d-3995-4d93-ade2-799b6b498867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:22.103782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb419d-3995-4d93-ade2-799b6b498867 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:22.104445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb419d-3995-4d93-ade2-799b6b498867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:22.105223 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb419d-3995-4d93-ade2-799b6b498867 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:22.106448 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb419d-3995-4d93-ade2-799b6b498867 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:09:22.106631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb419d-3995-4d93-ade2-799b6b498867 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:22.108676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb419d-3995-4d93-ade2-799b6b498867 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:22.117252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80881028-3c92-48d9-a003-174c8e05cc22 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:22.126697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 971/1941] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:09:22.126930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebecb9f-3fa0-4eba-bfd7-293cb989b2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:22.128064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebecb9f-3fa0-4eba-bfd7-293cb989b2eb service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:22.128400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebecb9f-3fa0-4eba-bfd7-293cb989b2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:22.128745 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebecb9f-3fa0-4eba-bfd7-293cb989b2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:22.133176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb419d-3995-4d93-ade2-799b6b498867 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:22.147255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebecb9f-3fa0-4eba-bfd7-293cb989b2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:22.149050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb419d-3995-4d93-ade2-799b6b498867 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:22.149192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebecb9f-3fa0-4eba-bfd7-293cb989b2eb service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:22.154637 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c541d-c5d7-4350-9a8e-061ab04a4493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:22.155309 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c541d-c5d7-4350-9a8e-061ab04a4493 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:22.155594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c541d-c5d7-4350-9a8e-061ab04a4493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:22.155877 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c541d-c5d7-4350-9a8e-061ab04a4493 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:22.156572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c541d-c5d7-4350-9a8e-061ab04a4493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:22.159012 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c541d-c5d7-4350-9a8e-061ab04a4493 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:22.161562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 971/1942] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:09:22.203182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 972/1943] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:22 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:22.204465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebecb9f-3fa0-4eba-bfd7-293cb989b2eb service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:22.219837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebecb9f-3fa0-4eba-bfd7-293cb989b2eb service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:22.299854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ed1b6-5475-4f49-8d81-99e0eadcd746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:22.300834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ed1b6-5475-4f49-8d81-99e0eadcd746 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:22.301127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ed1b6-5475-4f49-8d81-99e0eadcd746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:22.301411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ed1b6-5475-4f49-8d81-99e0eadcd746 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:22.302235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ed1b6-5475-4f49-8d81-99e0eadcd746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:22.304210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ed1b6-5475-4f49-8d81-99e0eadcd746 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:22.337476 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 972/1944] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:22 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:09:22.341517 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c541d-c5d7-4350-9a8e-061ab04a4493 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:22.360181 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c541d-c5d7-4350-9a8e-061ab04a4493 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:22.376477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2c88d-44fc-468a-a353-cfdfdbbec05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:22.377548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c88d-44fc-468a-a353-cfdfdbbec05e admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:22.379708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c88d-44fc-468a-a353-cfdfdbbec05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:22.381488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c88d-44fc-468a-a353-cfdfdbbec05e admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:22.385602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2c88d-44fc-468a-a353-cfdfdbbec05e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:22.390609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2c88d-44fc-468a-a353-cfdfdbbec05e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:22.435636 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 973/1945] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:22.435948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ed1b6-5475-4f49-8d81-99e0eadcd746 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:22.452977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ed1b6-5475-4f49-8d81-99e0eadcd746 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:22.463897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac63d9-77dd-45f8-b0b9-f32f7dbe6e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:22.464954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac63d9-77dd-45f8-b0b9-f32f7dbe6e1c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:22.465482 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac63d9-77dd-45f8-b0b9-f32f7dbe6e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:22.466169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac63d9-77dd-45f8-b0b9-f32f7dbe6e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:22.467616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac63d9-77dd-45f8-b0b9-f32f7dbe6e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:22.471169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac63d9-77dd-45f8-b0b9-f32f7dbe6e1c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:22.494427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 973/1946] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:09:22.494678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2c88d-44fc-468a-a353-cfdfdbbec05e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:22.507339 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2c88d-44fc-468a-a353-cfdfdbbec05e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:22.514912 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac21cc6-db99-46b9-ab1b-561062766d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:22.516224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac21cc6-db99-46b9-ab1b-561062766d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:22.516517 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac21cc6-db99-46b9-ab1b-561062766d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:22.517731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac21cc6-db99-46b9-ab1b-561062766d2d admin admin] PATH_INFO: `/v3/projects/d5ce573d72574211aeaf0581403a097c/users/cc7c3a085c1c4e5eb048153ba20b2fb0/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:22.522774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:22.543808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac21cc6-db99-46b9-ab1b-561062766d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ce573d72574211aeaf0581403a097c, user_id=cc7c3a085c1c4e5eb048153ba20b2fb0, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:22.547408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac21cc6-db99-46b9-ab1b-561062766d2d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:22.582753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 974/1947] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:22 2022] PUT /identity/v3/projects/d5ce573d72574211aeaf0581403a097c/users/cc7c3a085c1c4e5eb048153ba20b2fb0/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:22.589916 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc58210-ba0d-41fe-8ff3-050acbdbac66 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:22.592327 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc58210-ba0d-41fe-8ff3-050acbdbac66 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:22.592327 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc58210-ba0d-41fe-8ff3-050acbdbac66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:22.600206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:22.630709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac21cc6-db99-46b9-ab1b-561062766d2d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:22.633205 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4ac21cc6-db99-46b9-ab1b-561062766d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:22.661158 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-afc58210-ba0d-41fe-8ff3-050acbdbac66 None None] MFA Rules not processed for user `cc7c3a085c1c4e5eb048153ba20b2fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:22.665253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-afc58210-ba0d-41fe-8ff3-050acbdbac66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:22.779035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac21cc6-db99-46b9-ab1b-561062766d2d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:22.792864 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee9e16-c106-4884-83cc-b67cbb717e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:22.793676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee9e16-c106-4884-83cc-b67cbb717e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:22.793977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee9e16-c106-4884-83cc-b67cbb717e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:22.794257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee9e16-c106-4884-83cc-b67cbb717e3d admin admin] PATH_INFO: `/v3/projects/c5e1edbab5ce4b349ae054b107001758` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:22.800632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee9e16-c106-4884-83cc-b67cbb717e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e1edbab5ce4b349ae054b107001758)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:22.803591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee9e16-c106-4884-83cc-b67cbb717e3d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:22.818802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 974/1948] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:09:22.836638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc58210-ba0d-41fe-8ff3-050acbdbac66 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:22.838259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-afc58210-ba0d-41fe-8ff3-050acbdbac66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:22.917152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-91ee9e16-c106-4884-83cc-b67cbb717e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e1edbab5ce4b349ae054b107001758', 'request_id': 'req-91ee9e16-c106-4884-83cc-b67cbb717e3d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:22.936189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc58210-ba0d-41fe-8ff3-050acbdbac66 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:22.945887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-91ee9e16-c106-4884-83cc-b67cbb717e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e1edbab5ce4b349ae054b107001758', 'request_id': 'req-91ee9e16-c106-4884-83cc-b67cbb717e3d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:22.952885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-91ee9e16-c106-4884-83cc-b67cbb717e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e1edbab5ce4b349ae054b107001758', 'request_id': 'req-91ee9e16-c106-4884-83cc-b67cbb717e3d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:22.971692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-91ee9e16-c106-4884-83cc-b67cbb717e3d admin admin] The token cache is being invalidate because project c5e1edbab5ce4b349ae054b107001758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:22.971992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-91ee9e16-c106-4884-83cc-b67cbb717e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:22.972702 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-91ee9e16-c106-4884-83cc-b67cbb717e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:22.975779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 975/1949] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:22 2022] DELETE /identity/v3/projects/c5e1edbab5ce4b349ae054b107001758 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:09:22.980569 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794209e-4b25-4627-97f7-64916579abdb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:22.980809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794209e-4b25-4627-97f7-64916579abdb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:22.981146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794209e-4b25-4627-97f7-64916579abdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:22.999954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4794209e-4b25-4627-97f7-64916579abdb None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:23.000849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81fec2-56af-48a4-9827-a3a3079aa35d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:23.001247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4794209e-4b25-4627-97f7-64916579abdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:23.001544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81fec2-56af-48a4-9827-a3a3079aa35d service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:23.001937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81fec2-56af-48a4-9827-a3a3079aa35d service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:23.002276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81fec2-56af-48a4-9827-a3a3079aa35d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:23.004171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ea81fec2-56af-48a4-9827-a3a3079aa35d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:23.076217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 976/1950] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:09:23.079309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81fec2-56af-48a4-9827-a3a3079aa35d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:23.081265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81fec2-56af-48a4-9827-a3a3079aa35d service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:23.085039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ecd3c-b17e-41e6-95d7-d8687ffce00d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:23.085486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ecd3c-b17e-41e6-95d7-d8687ffce00d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:23.085888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ecd3c-b17e-41e6-95d7-d8687ffce00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:23.093819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 975/1951] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:23.126272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-fa8ecd3c-b17e-41e6-95d7-d8687ffce00d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:23.126272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fa8ecd3c-b17e-41e6-95d7-d8687ffce00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:23.137226 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 977/1952] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:23.141709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81fec2-56af-48a4-9827-a3a3079aa35d service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:23.147883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:23.154781 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81fec2-56af-48a4-9827-a3a3079aa35d service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:23.161157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757bd4b-6022-49e9-a888-6a785c67675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:23.161954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757bd4b-6022-49e9-a888-6a785c67675f admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:23.162312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757bd4b-6022-49e9-a888-6a785c67675f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:23.162679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757bd4b-6022-49e9-a888-6a785c67675f admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:23.163486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757bd4b-6022-49e9-a888-6a785c67675f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:09:23.163855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757bd4b-6022-49e9-a888-6a785c67675f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:23.165796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757bd4b-6022-49e9-a888-6a785c67675f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:23.181784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 976/1953] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:09:23.185422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ecd3c-b17e-41e6-95d7-d8687ffce00d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:23.195033 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ecd3c-b17e-41e6-95d7-d8687ffce00d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:23.200366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3dae0-5502-42da-818c-37bce738c7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:23.201072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3dae0-5502-42da-818c-37bce738c7f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:23.201649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3dae0-5502-42da-818c-37bce738c7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:23.201986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3dae0-5502-42da-818c-37bce738c7f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:23.202783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3dae0-5502-42da-818c-37bce738c7f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:23.205078 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3dae0-5502-42da-818c-37bce738c7f5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:23.210377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:23.247794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 978/1954] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:23 2022] POST /identity/v3/projects => generated 359 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:23.251192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757bd4b-6022-49e9-a888-6a785c67675f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:23.262047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757bd4b-6022-49e9-a888-6a785c67675f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:23.382583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4918be-e245-405e-bf2c-253b9f0af945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:23.384588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4918be-e245-405e-bf2c-253b9f0af945 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:23.384800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4918be-e245-405e-bf2c-253b9f0af945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:23.385609 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4918be-e245-405e-bf2c-253b9f0af945 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:23.385906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4918be-e245-405e-bf2c-253b9f0af945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:23.388876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4918be-e245-405e-bf2c-253b9f0af945 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:23.426150 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 977/1955] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:23 2022] POST /identity/v3/users => generated 280 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:23.426579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3dae0-5502-42da-818c-37bce738c7f5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:23.440066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3dae0-5502-42da-818c-37bce738c7f5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:23.445499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7b3ca-94ef-4b07-b6b3-067d4862ef2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:23.446408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7b3ca-94ef-4b07-b6b3-067d4862ef2f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:23.446788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7b3ca-94ef-4b07-b6b3-067d4862ef2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:23.447105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7b3ca-94ef-4b07-b6b3-067d4862ef2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:23.448322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7b3ca-94ef-4b07-b6b3-067d4862ef2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:23.449573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7b3ca-94ef-4b07-b6b3-067d4862ef2f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:23.464211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 979/1956] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:23.468475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4918be-e245-405e-bf2c-253b9f0af945 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:23.500871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4918be-e245-405e-bf2c-253b9f0af945 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:23.514239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100442a-2556-460a-beb2-f9bba654ad79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:23.515048 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100442a-2556-460a-beb2-f9bba654ad79 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:23.515434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100442a-2556-460a-beb2-f9bba654ad79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:23.515815 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100442a-2556-460a-beb2-f9bba654ad79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:23.516753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100442a-2556-460a-beb2-f9bba654ad79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:23.519432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100442a-2556-460a-beb2-f9bba654ad79 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:23.537666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 978/1957] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:23.541251 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7b3ca-94ef-4b07-b6b3-067d4862ef2f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:23.552949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7b3ca-94ef-4b07-b6b3-067d4862ef2f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:23.558119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd701e2-b993-4e67-96c3-ab1307ce5b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:23.559491 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd701e2-b993-4e67-96c3-ab1307ce5b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:23.559793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd701e2-b993-4e67-96c3-ab1307ce5b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:23.560255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd701e2-b993-4e67-96c3-ab1307ce5b3b admin admin] PATH_INFO: `/v3/projects/2145dbb5366b45c99cd1a8d4aa1e5f9e/users/22182a887af0482bb3597780e8d8574b/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:23.577996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd701e2-b993-4e67-96c3-ab1307ce5b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2145dbb5366b45c99cd1a8d4aa1e5f9e, user_id=22182a887af0482bb3597780e8d8574b, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:23.580950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd701e2-b993-4e67-96c3-ab1307ce5b3b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:23.597031 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 980/1958] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:23 2022] PUT /identity/v3/projects/2145dbb5366b45c99cd1a8d4aa1e5f9e/users/22182a887af0482bb3597780e8d8574b/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:23.609290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bf49f-3f21-46a9-b408-3cca1e176d4c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:23.609400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bf49f-3f21-46a9-b408-3cca1e176d4c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:23.609741 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bf49f-3f21-46a9-b408-3cca1e176d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:23.638545 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2f6bf49f-3f21-46a9-b408-3cca1e176d4c None None] MFA Rules not processed for user `22182a887af0482bb3597780e8d8574b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:23.639779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2f6bf49f-3f21-46a9-b408-3cca1e176d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:23.710281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 979/1959] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:23 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:23.749387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd701e2-b993-4e67-96c3-ab1307ce5b3b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:23.751286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-dbd701e2-b993-4e67-96c3-ab1307ce5b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:23.836883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd701e2-b993-4e67-96c3-ab1307ce5b3b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:23.841415 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250b686-04ed-446f-85b9-60f13f4d4233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:23.842512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250b686-04ed-446f-85b9-60f13f4d4233 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:23.843199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250b686-04ed-446f-85b9-60f13f4d4233 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:23.843784 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250b686-04ed-446f-85b9-60f13f4d4233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:23.869262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250b686-04ed-446f-85b9-60f13f4d4233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:23.870777 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250b686-04ed-446f-85b9-60f13f4d4233 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:23.904281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 981/1960] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:23 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:23.924013 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bf49f-3f21-46a9-b408-3cca1e176d4c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:23.977389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250b686-04ed-446f-85b9-60f13f4d4233 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:23.980099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3250b686-04ed-446f-85b9-60f13f4d4233 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:24.040191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:24.048759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bf49f-3f21-46a9-b408-3cca1e176d4c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:24.056267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9d4f1-ec0a-43fe-8564-d7563f14a0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:24.057276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9d4f1-ec0a-43fe-8564-d7563f14a0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:24.057567 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9d4f1-ec0a-43fe-8564-d7563f14a0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:24.057894 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9d4f1-ec0a-43fe-8564-d7563f14a0fd admin admin] PATH_INFO: `/v3/users/22182a887af0482bb3597780e8d8574b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:24.059133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9d4f1-ec0a-43fe-8564-d7563f14a0fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=22182a887af0482bb3597780e8d8574b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:24.061086 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9d4f1-ec0a-43fe-8564-d7563f14a0fd admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:24.085462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250b686-04ed-446f-85b9-60f13f4d4233 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:24.092971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad24ca-b9a8-4abc-a7e1-77bd0103049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:24.094006 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad24ca-b9a8-4abc-a7e1-77bd0103049d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:24.094937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad24ca-b9a8-4abc-a7e1-77bd0103049d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:24.094937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad24ca-b9a8-4abc-a7e1-77bd0103049d admin admin] PATH_INFO: `/v3/users/73a55e8e2585484c836048838c54c2e6` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:24.096564 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad24ca-b9a8-4abc-a7e1-77bd0103049d admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a55e8e2585484c836048838c54c2e6)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:24.098957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad24ca-b9a8-4abc-a7e1-77bd0103049d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:24.132225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-58d9d4f1-ec0a-43fe-8564-d7563f14a0fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22182a887af0482bb3597780e8d8574b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.132885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-58d9d4f1-ec0a-43fe-8564-d7563f14a0fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22182a887af0482bb3597780e8d8574b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.142318 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-58d9d4f1-ec0a-43fe-8564-d7563f14a0fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22182a887af0482bb3597780e8d8574b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.156243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-58d9d4f1-ec0a-43fe-8564-d7563f14a0fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22182a887af0482bb3597780e8d8574b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.157002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-58d9d4f1-ec0a-43fe-8564-d7563f14a0fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22182a887af0482bb3597780e8d8574b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.165287 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-92ad24ca-b9a8-4abc-a7e1-77bd0103049d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a55e8e2585484c836048838c54c2e6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.166155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-92ad24ca-b9a8-4abc-a7e1-77bd0103049d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a55e8e2585484c836048838c54c2e6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.171522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 980/1961] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:23 2022] DELETE /identity/v3/users/22182a887af0482bb3597780e8d8574b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:24.177215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-92ad24ca-b9a8-4abc-a7e1-77bd0103049d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a55e8e2585484c836048838c54c2e6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.177958 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96e1d7-d75d-48cf-a249-b2c0c82b1ed8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:24.178253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96e1d7-d75d-48cf-a249-b2c0c82b1ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:24.178535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96e1d7-d75d-48cf-a249-b2c0c82b1ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:24.179628 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:24.194219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-92ad24ca-b9a8-4abc-a7e1-77bd0103049d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a55e8e2585484c836048838c54c2e6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.195286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-92ad24ca-b9a8-4abc-a7e1-77bd0103049d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a55e8e2585484c836048838c54c2e6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.198753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ba96e1d7-d75d-48cf-a249-b2c0c82b1ed8 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:24.199955 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ba96e1d7-d75d-48cf-a249-b2c0c82b1ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:24.208488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 982/1962] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:23 2022] DELETE /identity/v3/users/73a55e8e2585484c836048838c54c2e6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:09:24.212880 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da4769-bb87-4f63-b31b-a04600bcc2b7 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:24.213172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da4769-bb87-4f63-b31b-a04600bcc2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:24.213543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da4769-bb87-4f63-b31b-a04600bcc2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:24.230747 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-71da4769-bb87-4f63-b31b-a04600bcc2b7 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:24.231985 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-71da4769-bb87-4f63-b31b-a04600bcc2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:24.259467 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 981/1963] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:09:24.270129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96e1d7-d75d-48cf-a249-b2c0c82b1ed8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:24.271519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ba96e1d7-d75d-48cf-a249-b2c0c82b1ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:24.292989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 983/1964] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:24.303133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da4769-bb87-4f63-b31b-a04600bcc2b7 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:24.304778 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-71da4769-bb87-4f63-b31b-a04600bcc2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:24.331480 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96e1d7-d75d-48cf-a249-b2c0c82b1ed8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:24.364623 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da4769-bb87-4f63-b31b-a04600bcc2b7 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:24.367284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:24.369611 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e01747-b67d-4994-9a65-71a5aba0c6fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:24.370714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e01747-b67d-4994-9a65-71a5aba0c6fe service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:24.371041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e01747-b67d-4994-9a65-71a5aba0c6fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:24.371299 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e01747-b67d-4994-9a65-71a5aba0c6fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:24.382622 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e01747-b67d-4994-9a65-71a5aba0c6fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:24.384315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e01747-b67d-4994-9a65-71a5aba0c6fe service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:24.393651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cca66e-c64d-4f65-b753-99b7d9058dba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:24.394471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cca66e-c64d-4f65-b753-99b7d9058dba service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:24.394902 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cca66e-c64d-4f65-b753-99b7d9058dba service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:24.395188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cca66e-c64d-4f65-b753-99b7d9058dba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:24.395398 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 984/1965] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:09:24.405751 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cca66e-c64d-4f65-b753-99b7d9058dba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:24.407206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cca66e-c64d-4f65-b753-99b7d9058dba service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:24.419743 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 982/1966] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:09:24.780861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e01747-b67d-4994-9a65-71a5aba0c6fe service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:24.782566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c7e01747-b67d-4994-9a65-71a5aba0c6fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:24.805577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e01747-b67d-4994-9a65-71a5aba0c6fe service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:24.832335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:24.837259 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:24.837649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:24.837879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] PATH_INFO: `/v3/projects/2145dbb5366b45c99cd1a8d4aa1e5f9e` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:24.840265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2145dbb5366b45c99cd1a8d4aa1e5f9e)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:24.846541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:24.934029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2145dbb5366b45c99cd1a8d4aa1e5f9e', 'request_id': 'req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.942180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2145dbb5366b45c99cd1a8d4aa1e5f9e', 'request_id': 'req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.955244 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2145dbb5366b45c99cd1a8d4aa1e5f9e', 'request_id': 'req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.968734 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:24.976237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] The token cache is being invalidate because project 2145dbb5366b45c99cd1a8d4aa1e5f9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:24.976913 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.978444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:24.981420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 985/1967] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:24 2022] DELETE /identity/v3/projects/2145dbb5366b45c99cd1a8d4aa1e5f9e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:24.986169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b409cc-583f-46d9-9067-df822ffda232 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:24.986960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b409cc-583f-46d9-9067-df822ffda232 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:24.986960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b409cc-583f-46d9-9067-df822ffda232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.007596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-43b409cc-583f-46d9-9067-df822ffda232 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:25.009052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-43b409cc-583f-46d9-9067-df822ffda232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:25.046750 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.048422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:25.102199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 983/1968] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:25.109844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3491bf0-fe39-4bcd-bd5a-60acb63f409e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.110143 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3491bf0-fe39-4bcd-bd5a-60acb63f409e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.110857 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3491bf0-fe39-4bcd-bd5a-60acb63f409e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.148594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f3491bf0-fe39-4bcd-bd5a-60acb63f409e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:25.150110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f3491bf0-fe39-4bcd-bd5a-60acb63f409e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:25.156436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b27a9a-a875-4e2f-9d91-8c4ebb813f0c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.160779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 984/1969] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:25.162704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb5336-50da-4cc5-ad62-a83bebfc3575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:25.163594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb5336-50da-4cc5-ad62-a83bebfc3575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.163880 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb5336-50da-4cc5-ad62-a83bebfc3575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.164161 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb5336-50da-4cc5-ad62-a83bebfc3575 admin admin] PATH_INFO: `/v3/projects/0e9d0fba002a490cbbd2759d5ecfbe75` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.165175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3491bf0-fe39-4bcd-bd5a-60acb63f409e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.165364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb5336-50da-4cc5-ad62-a83bebfc3575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e9d0fba002a490cbbd2759d5ecfbe75)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:25.167397 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb5336-50da-4cc5-ad62-a83bebfc3575 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:25.184138 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3491bf0-fe39-4bcd-bd5a-60acb63f409e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.195406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcdd31-71e4-4b80-b185-6ba7d71a99ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:25.196309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcdd31-71e4-4b80-b185-6ba7d71a99ad admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.196591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcdd31-71e4-4b80-b185-6ba7d71a99ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.196864 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcdd31-71e4-4b80-b185-6ba7d71a99ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.197606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcdd31-71e4-4b80-b185-6ba7d71a99ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:09:25.197955 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcdd31-71e4-4b80-b185-6ba7d71a99ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:25.199578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcdd31-71e4-4b80-b185-6ba7d71a99ad admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:25.214333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 985/1970] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:09:25.217205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcdd31-71e4-4b80-b185-6ba7d71a99ad admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.243086 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcdd31-71e4-4b80-b185-6ba7d71a99ad admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.254927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-96bb5336-50da-4cc5-ad62-a83bebfc3575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e9d0fba002a490cbbd2759d5ecfbe75', 'request_id': 'req-96bb5336-50da-4cc5-ad62-a83bebfc3575'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:25.262516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-96bb5336-50da-4cc5-ad62-a83bebfc3575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e9d0fba002a490cbbd2759d5ecfbe75', 'request_id': 'req-96bb5336-50da-4cc5-ad62-a83bebfc3575'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:25.270258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-96bb5336-50da-4cc5-ad62-a83bebfc3575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e9d0fba002a490cbbd2759d5ecfbe75', 'request_id': 'req-96bb5336-50da-4cc5-ad62-a83bebfc3575'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:25.294129 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-96bb5336-50da-4cc5-ad62-a83bebfc3575 admin admin] The token cache is being invalidate because project 0e9d0fba002a490cbbd2759d5ecfbe75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:25.295575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-96bb5336-50da-4cc5-ad62-a83bebfc3575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:25.296719 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-96bb5336-50da-4cc5-ad62-a83bebfc3575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:25.300156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 986/1971] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:25 2022] DELETE /identity/v3/projects/0e9d0fba002a490cbbd2759d5ecfbe75 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:25.305509 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e3cce-397c-4020-9f74-2ce7d9c83f5c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.305960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e3cce-397c-4020-9f74-2ce7d9c83f5c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.306342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e3cce-397c-4020-9f74-2ce7d9c83f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.333911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-172e3cce-397c-4020-9f74-2ce7d9c83f5c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:25.335277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-172e3cce-397c-4020-9f74-2ce7d9c83f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:25.345497 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad2297-14dd-4b05-9e35-c1e968e31222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:25.346644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad2297-14dd-4b05-9e35-c1e968e31222 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.346940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad2297-14dd-4b05-9e35-c1e968e31222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.347217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad2297-14dd-4b05-9e35-c1e968e31222 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.347958 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad2297-14dd-4b05-9e35-c1e968e31222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:25.349994 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad2297-14dd-4b05-9e35-c1e968e31222 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:25.393605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 986/1972] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:25 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:25.397368 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad2297-14dd-4b05-9e35-c1e968e31222 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.399450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d0ad2297-14dd-4b05-9e35-c1e968e31222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:25.421153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 987/1973] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:09:25.429064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c954606-a358-406f-8c23-d5c61c7bc577 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.429352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c954606-a358-406f-8c23-d5c61c7bc577 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.429661 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c954606-a358-406f-8c23-d5c61c7bc577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.448521 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4c954606-a358-406f-8c23-d5c61c7bc577 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:25.449774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4c954606-a358-406f-8c23-d5c61c7bc577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:25.457587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:25.477252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:25.486442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad2297-14dd-4b05-9e35-c1e968e31222 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.493649 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec2c8f-f6e9-4eca-83b2-2f000460a87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:25.494472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec2c8f-f6e9-4eca-83b2-2f000460a87f admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.494861 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec2c8f-f6e9-4eca-83b2-2f000460a87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.495233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec2c8f-f6e9-4eca-83b2-2f000460a87f admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.496683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec2c8f-f6e9-4eca-83b2-2f000460a87f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:25.498979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec2c8f-f6e9-4eca-83b2-2f000460a87f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:25.514767 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 988/1974] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:25.519101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c954606-a358-406f-8c23-d5c61c7bc577 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.529465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 987/1975] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:25 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:09:25.530283 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec2c8f-f6e9-4eca-83b2-2f000460a87f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.531396 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c954606-a358-406f-8c23-d5c61c7bc577 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.536127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91e9e9-5711-4abc-8fd3-47887a46fefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:25.536793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91e9e9-5711-4abc-8fd3-47887a46fefd admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.537071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91e9e9-5711-4abc-8fd3-47887a46fefd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.537781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91e9e9-5711-4abc-8fd3-47887a46fefd admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.538118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91e9e9-5711-4abc-8fd3-47887a46fefd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:09:25.538421 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91e9e9-5711-4abc-8fd3-47887a46fefd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:25.540020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91e9e9-5711-4abc-8fd3-47887a46fefd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:25.541671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec2c8f-f6e9-4eca-83b2-2f000460a87f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.548158 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069dc08-920c-4488-91a7-257d85a30fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:25.549782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069dc08-920c-4488-91a7-257d85a30fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.550319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069dc08-920c-4488-91a7-257d85a30fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.550516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069dc08-920c-4488-91a7-257d85a30fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.551260 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069dc08-920c-4488-91a7-257d85a30fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:25.552164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 989/1976] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 10:09:25.553160 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069dc08-920c-4488-91a7-257d85a30fd6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:25.557561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91e9e9-5711-4abc-8fd3-47887a46fefd admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.566789 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 988/1977] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:25.568134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91e9e9-5711-4abc-8fd3-47887a46fefd admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.570489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069dc08-920c-4488-91a7-257d85a30fd6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.573166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990c5b4-52db-44d8-87b6-af70f749889a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:25.573927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990c5b4-52db-44d8-87b6-af70f749889a admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.574273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990c5b4-52db-44d8-87b6-af70f749889a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.574636 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990c5b4-52db-44d8-87b6-af70f749889a admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.575380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990c5b4-52db-44d8-87b6-af70f749889a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:25.577419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990c5b4-52db-44d8-87b6-af70f749889a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:25.582315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069dc08-920c-4488-91a7-257d85a30fd6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.588541 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4ac8b-2d25-4d38-a02c-e562ac825c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:25.589239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4ac8b-2d25-4d38-a02c-e562ac825c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.589719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4ac8b-2d25-4d38-a02c-e562ac825c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.590003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4ac8b-2d25-4d38-a02c-e562ac825c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.590738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4ac8b-2d25-4d38-a02c-e562ac825c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:25.592387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4ac8b-2d25-4d38-a02c-e562ac825c81 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:25.604779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 989/1978] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:09:25.608860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4ac8b-2d25-4d38-a02c-e562ac825c81 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.609168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 990/1979] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:25 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:09:25.613843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990c5b4-52db-44d8-87b6-af70f749889a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.623612 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4ac8b-2d25-4d38-a02c-e562ac825c81 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.700409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990c5b4-52db-44d8-87b6-af70f749889a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.700631 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e7626-5545-454d-8ad7-2de2dbc5402d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:25.701373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e7626-5545-454d-8ad7-2de2dbc5402d admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.701696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e7626-5545-454d-8ad7-2de2dbc5402d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.701977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e7626-5545-454d-8ad7-2de2dbc5402d admin admin] PATH_INFO: `/v3/projects/59fcbc120f0d4dc48dc0b3f2e7433b1a/users/348d6a68d7c84200899be9e72aeaded6/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.707346 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae16d38-bf9a-46a1-8737-4e7a7206bb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:25.708206 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae16d38-bf9a-46a1-8737-4e7a7206bb45 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.708648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae16d38-bf9a-46a1-8737-4e7a7206bb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.709150 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae16d38-bf9a-46a1-8737-4e7a7206bb45 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.710122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae16d38-bf9a-46a1-8737-4e7a7206bb45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:25.712282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae16d38-bf9a-46a1-8737-4e7a7206bb45 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:25.722751 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e7626-5545-454d-8ad7-2de2dbc5402d admin admin] RBAC: Authorizing `identity:create_grant(project_id=59fcbc120f0d4dc48dc0b3f2e7433b1a, user_id=348d6a68d7c84200899be9e72aeaded6, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:25.727671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e7626-5545-454d-8ad7-2de2dbc5402d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:25.742130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 991/1980] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:25 2022] POST /identity/v3/users => generated 282 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:09:25.745902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae16d38-bf9a-46a1-8737-4e7a7206bb45 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.752070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 990/1981] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:25 2022] PUT /identity/v3/projects/59fcbc120f0d4dc48dc0b3f2e7433b1a/users/348d6a68d7c84200899be9e72aeaded6/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:09:25.758203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae16d38-bf9a-46a1-8737-4e7a7206bb45 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.760626 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5016bd6-becc-4a63-ae30-378bed75935b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.761113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5016bd6-becc-4a63-ae30-378bed75935b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.761569 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5016bd6-becc-4a63-ae30-378bed75935b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.797955 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a5016bd6-becc-4a63-ae30-378bed75935b None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:25.799395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a5016bd6-becc-4a63-ae30-378bed75935b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:25.841672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6ea4b-b7df-4e95-a725-1f6bac4e1a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:25.842450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6ea4b-b7df-4e95-a725-1f6bac4e1a22 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.842961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6ea4b-b7df-4e95-a725-1f6bac4e1a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.843301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6ea4b-b7df-4e95-a725-1f6bac4e1a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.844885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6ea4b-b7df-4e95-a725-1f6bac4e1a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:25.846751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6ea4b-b7df-4e95-a725-1f6bac4e1a22 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:25.861397 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 992/1982] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:09:25.862991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6ea4b-b7df-4e95-a725-1f6bac4e1a22 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.880235 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6ea4b-b7df-4e95-a725-1f6bac4e1a22 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.888810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 991/1983] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:09:25.891853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce128272-6c7f-4c4c-bf1d-0f00d36bfc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:25.893201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce128272-6c7f-4c4c-bf1d-0f00d36bfc8e admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.893918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce128272-6c7f-4c4c-bf1d-0f00d36bfc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.894195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce128272-6c7f-4c4c-bf1d-0f00d36bfc8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.895650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce128272-6c7f-4c4c-bf1d-0f00d36bfc8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:25.897213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5016bd6-becc-4a63-ae30-378bed75935b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.898107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce128272-6c7f-4c4c-bf1d-0f00d36bfc8e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:25.898823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a5016bd6-becc-4a63-ae30-378bed75935b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:25.916789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 993/1984] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:25.920220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce128272-6c7f-4c4c-bf1d-0f00d36bfc8e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:25.930876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce128272-6c7f-4c4c-bf1d-0f00d36bfc8e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.935529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967fbf2-a35c-4246-980e-3c272e5ca172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:25.936412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967fbf2-a35c-4246-980e-3c272e5ca172 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.936697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967fbf2-a35c-4246-980e-3c272e5ca172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.936976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967fbf2-a35c-4246-980e-3c272e5ca172 admin admin] PATH_INFO: `/v3/projects/846253ae8e114af4a874751ca1f70936/users/1e873cc6aea340939f142af452be6ae3/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:25.946507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:25.946724 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:25.954072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967fbf2-a35c-4246-980e-3c272e5ca172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=846253ae8e114af4a874751ca1f70936, user_id=1e873cc6aea340939f142af452be6ae3, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:25.957766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967fbf2-a35c-4246-980e-3c272e5ca172 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:25.973778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5016bd6-becc-4a63-ae30-378bed75935b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:25.975165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 994/1985] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:25 2022] PUT /identity/v3/projects/846253ae8e114af4a874751ca1f70936/users/1e873cc6aea340939f142af452be6ae3/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:09:25.986734 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adfb59-d0c9-4641-9bc8-a5a6bb721a04 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:25.987086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adfb59-d0c9-4641-9bc8-a5a6bb721a04 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:25.987383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adfb59-d0c9-4641-9bc8-a5a6bb721a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:26.014903 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-18adfb59-d0c9-4641-9bc8-a5a6bb721a04 None None] MFA Rules not processed for user `1e873cc6aea340939f142af452be6ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:26.016120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-18adfb59-d0c9-4641-9bc8-a5a6bb721a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:26.035798 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfed0c-21e7-4a75-9ce8-bc936883f3eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:26.037549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfed0c-21e7-4a75-9ce8-bc936883f3eb service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:26.037549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfed0c-21e7-4a75-9ce8-bc936883f3eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:26.037549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfed0c-21e7-4a75-9ce8-bc936883f3eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:26.088525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 995/1986] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:25 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:09:26.100775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adfb59-d0c9-4641-9bc8-a5a6bb721a04 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:26.102254 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-18adfb59-d0c9-4641-9bc8-a5a6bb721a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:26.110669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfed0c-21e7-4a75-9ce8-bc936883f3eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:26.112264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfed0c-21e7-4a75-9ce8-bc936883f3eb service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:26.125821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 992/1987] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:09:26.180139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adfb59-d0c9-4641-9bc8-a5a6bb721a04 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:26.184315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31ce1b-75d3-4298-ad68-1bdd782aff39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:26.185100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31ce1b-75d3-4298-ad68-1bdd782aff39 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:26.185392 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31ce1b-75d3-4298-ad68-1bdd782aff39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:26.185705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31ce1b-75d3-4298-ad68-1bdd782aff39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:26.196798 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31ce1b-75d3-4298-ad68-1bdd782aff39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:26.198322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31ce1b-75d3-4298-ad68-1bdd782aff39 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:26.208905 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 996/1988] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:26 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:09:27.152355 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b4e86-5f04-4146-ac31-dc3c264f728d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:27.152911 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b4e86-5f04-4146-ac31-dc3c264f728d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:27.153149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b4e86-5f04-4146-ac31-dc3c264f728d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:27.184321 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-785b4e86-5f04-4146-ac31-dc3c264f728d None None] MFA Rules not processed for user `348d6a68d7c84200899be9e72aeaded6`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:27.185826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-785b4e86-5f04-4146-ac31-dc3c264f728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:27.259218 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 993/1989] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:27.270768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31ce1b-75d3-4298-ad68-1bdd782aff39 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:27.272790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7e31ce1b-75d3-4298-ad68-1bdd782aff39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:27.285113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31ce1b-75d3-4298-ad68-1bdd782aff39 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:27.288731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260af24-d086-4b1d-8bcc-9915c9643bf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:27.289587 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260af24-d086-4b1d-8bcc-9915c9643bf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:27.289897 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260af24-d086-4b1d-8bcc-9915c9643bf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:27.290176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260af24-d086-4b1d-8bcc-9915c9643bf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:27.300202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260af24-d086-4b1d-8bcc-9915c9643bf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:27.301371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260af24-d086-4b1d-8bcc-9915c9643bf0 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:27.322508 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 997/1990] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:28.694445 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c379d2-7a59-44c6-a31e-f5d2c8f3fecb None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:28.695017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c379d2-7a59-44c6-a31e-f5d2c8f3fecb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:28.695017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c379d2-7a59-44c6-a31e-f5d2c8f3fecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:28.714772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-70c379d2-7a59-44c6-a31e-f5d2c8f3fecb None None] MFA Rules not processed for user `348d6a68d7c84200899be9e72aeaded6`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:28.716009 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-70c379d2-7a59-44c6-a31e-f5d2c8f3fecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:28.727928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 994/1991] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:37.307138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=69,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:38.709371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:47.896443 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260af24-d086-4b1d-8bcc-9915c9643bf0 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:47.898124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3260af24-d086-4b1d-8bcc-9915c9643bf0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:47.937286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260af24-d086-4b1d-8bcc-9915c9643bf0 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:47.941845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:47.942737 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:47.943018 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:47.943298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] PATH_INFO: `/v3/users/cc7c3a085c1c4e5eb048153ba20b2fb0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:47.944486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc7c3a085c1c4e5eb048153ba20b2fb0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:47.946482 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:48.004315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc7c3a085c1c4e5eb048153ba20b2fb0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:48.005200 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc7c3a085c1c4e5eb048153ba20b2fb0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:48.017020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc7c3a085c1c4e5eb048153ba20b2fb0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:48.032568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc7c3a085c1c4e5eb048153ba20b2fb0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:48.033513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc7c3a085c1c4e5eb048153ba20b2fb0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:48.049863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 998/1992] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:47 2022] DELETE /identity/v3/users/cc7c3a085c1c4e5eb048153ba20b2fb0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:48.053347 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573deba-2a94-4ddf-8721-10a0412f26b9 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:48.053858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573deba-2a94-4ddf-8721-10a0412f26b9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:48.053945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573deba-2a94-4ddf-8721-10a0412f26b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:48.072036 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d573deba-2a94-4ddf-8721-10a0412f26b9 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:48.073756 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d573deba-2a94-4ddf-8721-10a0412f26b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:48.146813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 995/1993] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:48.157864 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:48.159447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:48.218502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c6bd2-0c96-46fe-b672-66389bc21624 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:48.223277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d22b38-7485-462a-9f30-9fed8771baed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:48.223937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d22b38-7485-462a-9f30-9fed8771baed service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:48.224220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d22b38-7485-462a-9f30-9fed8771baed service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:48.224492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d22b38-7485-462a-9f30-9fed8771baed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:48.234323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d22b38-7485-462a-9f30-9fed8771baed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:48.235775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d22b38-7485-462a-9f30-9fed8771baed service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:48.245492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 999/1994] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:48.527284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573deba-2a94-4ddf-8721-10a0412f26b9 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:48.528741 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d573deba-2a94-4ddf-8721-10a0412f26b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:48.543268 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573deba-2a94-4ddf-8721-10a0412f26b9 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:48.549339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:48.550411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:48.550694 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:48.550976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c admin admin] PATH_INFO: `/v3/projects/d5ce573d72574211aeaf0581403a097c` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:48.557782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5ce573d72574211aeaf0581403a097c)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:48.559740 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:48.614266 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ce573d72574211aeaf0581403a097c', 'request_id': 'req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:48.621184 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ce573d72574211aeaf0581403a097c', 'request_id': 'req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:48.626762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ce573d72574211aeaf0581403a097c', 'request_id': 'req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:48.641483 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c admin admin] The token cache is being invalidate because project d5ce573d72574211aeaf0581403a097c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:48.641817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:48.642455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8e55d6ce-ece2-412c-aaf6-ae01eabaa94c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:48.644105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 996/1995] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:48 2022] DELETE /identity/v3/projects/d5ce573d72574211aeaf0581403a097c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:48.651896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b62a7a-19c6-4481-9ddb-0ca28ac5935b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:48.652193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b62a7a-19c6-4481-9ddb-0ca28ac5935b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:48.652472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b62a7a-19c6-4481-9ddb-0ca28ac5935b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:48.670615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c7b62a7a-19c6-4481-9ddb-0ca28ac5935b None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:48.671852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c7b62a7a-19c6-4481-9ddb-0ca28ac5935b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:48.738053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1000/1996] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:48.746015 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c39c69-4612-4675-8ef4-10a7c30e841c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:48.746307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c39c69-4612-4675-8ef4-10a7c30e841c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:48.746587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c39c69-4612-4675-8ef4-10a7c30e841c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:48.768081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-07c39c69-4612-4675-8ef4-10a7c30e841c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:48.769848 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-07c39c69-4612-4675-8ef4-10a7c30e841c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:48.783668 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 997/1997] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:48.788654 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b62a7a-19c6-4481-9ddb-0ca28ac5935b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:48.799241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b62a7a-19c6-4481-9ddb-0ca28ac5935b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:48.804393 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559b8b5-f188-492c-ab83-84abdd18a4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:48.805107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559b8b5-f188-492c-ab83-84abdd18a4ea admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:48.805384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559b8b5-f188-492c-ab83-84abdd18a4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:48.805648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559b8b5-f188-492c-ab83-84abdd18a4ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:48.806427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559b8b5-f188-492c-ab83-84abdd18a4ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:09:48.806730 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559b8b5-f188-492c-ab83-84abdd18a4ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:48.808380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559b8b5-f188-492c-ab83-84abdd18a4ea admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:48.820501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1001/1998] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:09:48.825246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c39c69-4612-4675-8ef4-10a7c30e841c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:48.835030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c39c69-4612-4675-8ef4-10a7c30e841c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:48.841035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e067b5-5d89-4559-9f10-0796b9cf4c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:48.841845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e067b5-5d89-4559-9f10-0796b9cf4c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:48.842145 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e067b5-5d89-4559-9f10-0796b9cf4c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:48.842422 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e067b5-5d89-4559-9f10-0796b9cf4c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:48.843123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e067b5-5d89-4559-9f10-0796b9cf4c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:48.845478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e067b5-5d89-4559-9f10-0796b9cf4c82 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:48.876168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 998/1999] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:48 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:48.880535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559b8b5-f188-492c-ab83-84abdd18a4ea admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:48.885884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:48.892933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559b8b5-f188-492c-ab83-84abdd18a4ea admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:48.959591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da927b-866c-47a2-a027-9315f0df571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:48.960285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da927b-866c-47a2-a027-9315f0df571c admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:48.960572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da927b-866c-47a2-a027-9315f0df571c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:48.960854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da927b-866c-47a2-a027-9315f0df571c admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:48.961557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da927b-866c-47a2-a027-9315f0df571c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:48.963561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da927b-866c-47a2-a027-9315f0df571c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:48.990310 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1002/2000] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:48 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:48.994414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e067b5-5d89-4559-9f10-0796b9cf4c82 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:49.005541 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e067b5-5d89-4559-9f10-0796b9cf4c82 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:49.010261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99528ad-3dfe-4e54-84f8-ee4bc4fb2143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:49.011007 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99528ad-3dfe-4e54-84f8-ee4bc4fb2143 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.011358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99528ad-3dfe-4e54-84f8-ee4bc4fb2143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.011777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99528ad-3dfe-4e54-84f8-ee4bc4fb2143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.013023 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99528ad-3dfe-4e54-84f8-ee4bc4fb2143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:49.014731 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99528ad-3dfe-4e54-84f8-ee4bc4fb2143 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:49.025609 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 999/2001] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:49.029834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da927b-866c-47a2-a027-9315f0df571c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:49.039960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da927b-866c-47a2-a027-9315f0df571c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:49.045155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487483b-2f39-47bf-b637-16682bc63b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:49.046015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487483b-2f39-47bf-b637-16682bc63b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.046303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487483b-2f39-47bf-b637-16682bc63b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.046580 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487483b-2f39-47bf-b637-16682bc63b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.047319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487483b-2f39-47bf-b637-16682bc63b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:49.048973 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487483b-2f39-47bf-b637-16682bc63b8f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:49.060359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1003/2002] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:49.064451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99528ad-3dfe-4e54-84f8-ee4bc4fb2143 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:49.074515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99528ad-3dfe-4e54-84f8-ee4bc4fb2143 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:49.079168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ce7a6-234e-4552-b6ae-15775aa43c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:49.080129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ce7a6-234e-4552-b6ae-15775aa43c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.080495 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ce7a6-234e-4552-b6ae-15775aa43c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.080833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ce7a6-234e-4552-b6ae-15775aa43c12 admin admin] PATH_INFO: `/v3/projects/1a4306d15f99496785653e78a488f3ea/users/581c575be77a4daeaa6fd5e20f3befdc/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.097137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ce7a6-234e-4552-b6ae-15775aa43c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a4306d15f99496785653e78a488f3ea, user_id=581c575be77a4daeaa6fd5e20f3befdc, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:49.100583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ce7a6-234e-4552-b6ae-15775aa43c12 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:49.119211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1000/2003] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:49 2022] PUT /identity/v3/projects/1a4306d15f99496785653e78a488f3ea/users/581c575be77a4daeaa6fd5e20f3befdc/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:09:49.132291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4488ed-74c3-48c2-a1e1-89995fd2bf2f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.132623 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4488ed-74c3-48c2-a1e1-89995fd2bf2f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.133377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4488ed-74c3-48c2-a1e1-89995fd2bf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.162524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-af4488ed-74c3-48c2-a1e1-89995fd2bf2f None None] MFA Rules not processed for user `581c575be77a4daeaa6fd5e20f3befdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:49.163770 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-af4488ed-74c3-48c2-a1e1-89995fd2bf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:49.232979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1004/2004] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:49.237433 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ce7a6-234e-4552-b6ae-15775aa43c12 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:49.260854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:49.316210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ce7a6-234e-4552-b6ae-15775aa43c12 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:49.321474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0f4f5-5acb-4e5b-aa0c-d88325235262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:49.322238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0f4f5-5acb-4e5b-aa0c-d88325235262 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.322522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0f4f5-5acb-4e5b-aa0c-d88325235262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.322799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0f4f5-5acb-4e5b-aa0c-d88325235262 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.323502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0f4f5-5acb-4e5b-aa0c-d88325235262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:49.325703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0f4f5-5acb-4e5b-aa0c-d88325235262 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:49.352681 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1001/2005] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:49 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:49.357017 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4488ed-74c3-48c2-a1e1-89995fd2bf2f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:49.368096 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4488ed-74c3-48c2-a1e1-89995fd2bf2f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:49.426513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a380f-cb53-41b5-9006-39eed2d9f9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:49.427094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a380f-cb53-41b5-9006-39eed2d9f9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.427382 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a380f-cb53-41b5-9006-39eed2d9f9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.427658 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a380f-cb53-41b5-9006-39eed2d9f9b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.428362 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a380f-cb53-41b5-9006-39eed2d9f9b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:49.430344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a380f-cb53-41b5-9006-39eed2d9f9b8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:49.448242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:49.458991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1005/2006] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:49 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:49.460234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0f4f5-5acb-4e5b-aa0c-d88325235262 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:49.471374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0f4f5-5acb-4e5b-aa0c-d88325235262 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:49.476435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257efabf-740a-4529-a0ed-575fb4f1ae14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:49.477098 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257efabf-740a-4529-a0ed-575fb4f1ae14 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.477378 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257efabf-740a-4529-a0ed-575fb4f1ae14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.477686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257efabf-740a-4529-a0ed-575fb4f1ae14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.478418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257efabf-740a-4529-a0ed-575fb4f1ae14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:49.480028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257efabf-740a-4529-a0ed-575fb4f1ae14 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:49.491008 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1002/2007] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:49.495253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a380f-cb53-41b5-9006-39eed2d9f9b8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:49.506716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a380f-cb53-41b5-9006-39eed2d9f9b8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:49.512254 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00731783-69fa-4ca0-b263-44280c9347da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:49.512988 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00731783-69fa-4ca0-b263-44280c9347da admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.513265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00731783-69fa-4ca0-b263-44280c9347da admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.513546 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00731783-69fa-4ca0-b263-44280c9347da admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.514314 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00731783-69fa-4ca0-b263-44280c9347da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:49.515941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00731783-69fa-4ca0-b263-44280c9347da admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:49.526041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1006/2008] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:49.531010 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257efabf-740a-4529-a0ed-575fb4f1ae14 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:49.544720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257efabf-740a-4529-a0ed-575fb4f1ae14 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:49.549713 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44852479-f01e-4285-ab51-65c3628f97db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:49.550594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44852479-f01e-4285-ab51-65c3628f97db admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.550870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44852479-f01e-4285-ab51-65c3628f97db admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.551160 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44852479-f01e-4285-ab51-65c3628f97db admin admin] PATH_INFO: `/v3/projects/ca947389b8564b3a9a4f8c6d319872a0/users/1da3714c65cc468fbe717dcacc643bfa/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.568060 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44852479-f01e-4285-ab51-65c3628f97db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca947389b8564b3a9a4f8c6d319872a0, user_id=1da3714c65cc468fbe717dcacc643bfa, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:49.570894 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44852479-f01e-4285-ab51-65c3628f97db admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:49.585946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1003/2009] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:49 2022] PUT /identity/v3/projects/ca947389b8564b3a9a4f8c6d319872a0/users/1da3714c65cc468fbe717dcacc643bfa/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:49.597717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26251c2-66ca-4db3-bcaa-31f136b6f08a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.598099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26251c2-66ca-4db3-bcaa-31f136b6f08a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.598372 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26251c2-66ca-4db3-bcaa-31f136b6f08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.625476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b26251c2-66ca-4db3-bcaa-31f136b6f08a None None] MFA Rules not processed for user `1da3714c65cc468fbe717dcacc643bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:49.626733 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b26251c2-66ca-4db3-bcaa-31f136b6f08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:49.698945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1007/2010] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:49.703698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44852479-f01e-4285-ab51-65c3628f97db admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:49.772269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44852479-f01e-4285-ab51-65c3628f97db admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:49.777669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951235f7-96c1-4ad1-b8c9-99aaaf467815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:49.778399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951235f7-96c1-4ad1-b8c9-99aaaf467815 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.778700 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951235f7-96c1-4ad1-b8c9-99aaaf467815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.778986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951235f7-96c1-4ad1-b8c9-99aaaf467815 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.779680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951235f7-96c1-4ad1-b8c9-99aaaf467815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:49.781606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951235f7-96c1-4ad1-b8c9-99aaaf467815 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:49.811171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1004/2011] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:09:49 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:49.815646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26251c2-66ca-4db3-bcaa-31f136b6f08a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:49.827470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26251c2-66ca-4db3-bcaa-31f136b6f08a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:49.889365 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca91bd-4fda-435a-8863-fb314d1a404f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:49.890068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca91bd-4fda-435a-8863-fb314d1a404f admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.890361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca91bd-4fda-435a-8863-fb314d1a404f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.890769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca91bd-4fda-435a-8863-fb314d1a404f admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.891464 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca91bd-4fda-435a-8863-fb314d1a404f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:49.893435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca91bd-4fda-435a-8863-fb314d1a404f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:49.915523 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1008/2012] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:09:49 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:09:49.919870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951235f7-96c1-4ad1-b8c9-99aaaf467815 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:49.930596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951235f7-96c1-4ad1-b8c9-99aaaf467815 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:49.936292 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf943c-6489-487d-a0c1-65310bd58c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:49.936954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf943c-6489-487d-a0c1-65310bd58c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.937232 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf943c-6489-487d-a0c1-65310bd58c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.937503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf943c-6489-487d-a0c1-65310bd58c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.938270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf943c-6489-487d-a0c1-65310bd58c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:49.939874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf943c-6489-487d-a0c1-65310bd58c81 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:49.950808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1005/2013] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:49.955070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca91bd-4fda-435a-8863-fb314d1a404f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:49.965086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca91bd-4fda-435a-8863-fb314d1a404f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:49.970641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80c431-aeff-448c-a7e8-ea54eb25d1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:49.971486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80c431-aeff-448c-a7e8-ea54eb25d1a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:49.971802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80c431-aeff-448c-a7e8-ea54eb25d1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:49.972043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80c431-aeff-448c-a7e8-ea54eb25d1a2 admin admin] PATH_INFO: `/v3/projects/756486e047be4714a5827ce01668ddae/users/cfa4c4f41d4a47d98a9c3af65eab2b35/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:49.987287 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80c431-aeff-448c-a7e8-ea54eb25d1a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=756486e047be4714a5827ce01668ddae, user_id=cfa4c4f41d4a47d98a9c3af65eab2b35, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:49.989974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80c431-aeff-448c-a7e8-ea54eb25d1a2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:50.008089 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1009/2014] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:49 2022] PUT /identity/v3/projects/756486e047be4714a5827ce01668ddae/users/cfa4c4f41d4a47d98a9c3af65eab2b35/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:50.012081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf943c-6489-487d-a0c1-65310bd58c81 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:50.023839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf943c-6489-487d-a0c1-65310bd58c81 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:50.066618 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:50.082151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38067a-f32e-47cc-9b38-33f560bda698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:50.082824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38067a-f32e-47cc-9b38-33f560bda698 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:50.083114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38067a-f32e-47cc-9b38-33f560bda698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:50.083394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38067a-f32e-47cc-9b38-33f560bda698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:50.084111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38067a-f32e-47cc-9b38-33f560bda698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:50.085757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38067a-f32e-47cc-9b38-33f560bda698 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:50.095837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1006/2015] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:09:50.100270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80c431-aeff-448c-a7e8-ea54eb25d1a2 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:50.111138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80c431-aeff-448c-a7e8-ea54eb25d1a2 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:50.116985 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2cedb-7f74-4a01-9942-f1f919a1763f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:50.117880 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2cedb-7f74-4a01-9942-f1f919a1763f admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:50.118165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2cedb-7f74-4a01-9942-f1f919a1763f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:50.118427 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2cedb-7f74-4a01-9942-f1f919a1763f admin admin] PATH_INFO: `/v3/projects/756486e047be4714a5827ce01668ddae/users/cfa4c4f41d4a47d98a9c3af65eab2b35/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:50.121068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2cedb-7f74-4a01-9942-f1f919a1763f admin admin] RBAC: Authorizing `identity:create_grant(project_id=756486e047be4714a5827ce01668ddae, user_id=cfa4c4f41d4a47d98a9c3af65eab2b35, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:50.124070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2cedb-7f74-4a01-9942-f1f919a1763f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:50.140166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1010/2016] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:09:50 2022] PUT /identity/v3/projects/756486e047be4714a5827ce01668ddae/users/cfa4c4f41d4a47d98a9c3af65eab2b35/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:50.151422 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d86756-1590-4ef5-b2be-137db8feb2f2 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:50.151715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d86756-1590-4ef5-b2be-137db8feb2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:50.152001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d86756-1590-4ef5-b2be-137db8feb2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:50.180679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e7d86756-1590-4ef5-b2be-137db8feb2f2 None None] MFA Rules not processed for user `cfa4c4f41d4a47d98a9c3af65eab2b35`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:50.181969 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e7d86756-1590-4ef5-b2be-137db8feb2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:50.252843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1007/2017] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:50.264625 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2cedb-7f74-4a01-9942-f1f919a1763f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:50.266561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3fe2cedb-7f74-4a01-9942-f1f919a1763f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:50.302330 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:50.327105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2cedb-7f74-4a01-9942-f1f919a1763f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:50.331280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7486f8c-2d6e-49ea-98b4-9a5864f2a816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:50.331926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7486f8c-2d6e-49ea-98b4-9a5864f2a816 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:50.332207 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7486f8c-2d6e-49ea-98b4-9a5864f2a816 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:50.332572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7486f8c-2d6e-49ea-98b4-9a5864f2a816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:50.402251 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7486f8c-2d6e-49ea-98b4-9a5864f2a816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:50.403989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7486f8c-2d6e-49ea-98b4-9a5864f2a816 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:50.413609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1011/2018] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:09:51.973373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d86756-1590-4ef5-b2be-137db8feb2f2 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:51.975016 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e7d86756-1590-4ef5-b2be-137db8feb2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:51.988733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d86756-1590-4ef5-b2be-137db8feb2f2 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:51.993410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150b1ff-61f0-4071-846f-421ca78fc7a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:51.994375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150b1ff-61f0-4071-846f-421ca78fc7a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:51.994669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150b1ff-61f0-4071-846f-421ca78fc7a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:51.995098 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150b1ff-61f0-4071-846f-421ca78fc7a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:52.056909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150b1ff-61f0-4071-846f-421ca78fc7a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:52.058547 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150b1ff-61f0-4071-846f-421ca78fc7a8 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:52.068212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1008/2019] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:52.825474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7486f8c-2d6e-49ea-98b4-9a5864f2a816 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:52.833899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7486f8c-2d6e-49ea-98b4-9a5864f2a816 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:52.838216 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0366c80-3ce2-49f7-8871-c1b0e4f2cb6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:52.838903 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0366c80-3ce2-49f7-8871-c1b0e4f2cb6f service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:52.839188 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0366c80-3ce2-49f7-8871-c1b0e4f2cb6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:52.839511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0366c80-3ce2-49f7-8871-c1b0e4f2cb6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:52.849555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0366c80-3ce2-49f7-8871-c1b0e4f2cb6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:52.851097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0366c80-3ce2-49f7-8871-c1b0e4f2cb6f service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:52.860823 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1012/2020] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:56.808406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150b1ff-61f0-4071-846f-421ca78fc7a8 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:56.880037 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150b1ff-61f0-4071-846f-421ca78fc7a8 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:56.885532 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:56.886446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:56.886732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:56.887008 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] PATH_INFO: `/v3/users/581c575be77a4daeaa6fd5e20f3befdc` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:56.888339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=581c575be77a4daeaa6fd5e20f3befdc)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:56.890345 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:56.895289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:56.941939 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '581c575be77a4daeaa6fd5e20f3befdc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:56.942629 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '581c575be77a4daeaa6fd5e20f3befdc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:56.951901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '581c575be77a4daeaa6fd5e20f3befdc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:56.964593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '581c575be77a4daeaa6fd5e20f3befdc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:56.965242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '581c575be77a4daeaa6fd5e20f3befdc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:56.978354 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1009/2021] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:56 2022] DELETE /identity/v3/users/581c575be77a4daeaa6fd5e20f3befdc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:56.983269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c89b38-55e5-4453-939a-05a6d2bea319 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:56.983739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c89b38-55e5-4453-939a-05a6d2bea319 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:56.983739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c89b38-55e5-4453-939a-05a6d2bea319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:57.001353 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-43c89b38-55e5-4453-939a-05a6d2bea319 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:57.002870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-43c89b38-55e5-4453-939a-05a6d2bea319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:57.065562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1013/2022] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:57.076962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:57.078632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:57.141533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14429f0-e51b-4ead-8596-eb01722558a3 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:57.145946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e77c6-8c0d-4857-8eef-eaf36894e9d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:57.146610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e77c6-8c0d-4857-8eef-eaf36894e9d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:57.146892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e77c6-8c0d-4857-8eef-eaf36894e9d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:57.147166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e77c6-8c0d-4857-8eef-eaf36894e9d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:57.153423 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6865826-1185-4596-b6cf-b12e32cf3c98 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:57.153750 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6865826-1185-4596-b6cf-b12e32cf3c98 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:57.154278 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6865826-1185-4596-b6cf-b12e32cf3c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:57.157723 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e77c6-8c0d-4857-8eef-eaf36894e9d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:57.159406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e77c6-8c0d-4857-8eef-eaf36894e9d1 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:57.169118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1010/2023] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:09:57.177065 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d6865826-1185-4596-b6cf-b12e32cf3c98 None None] MFA Rules not processed for user `348d6a68d7c84200899be9e72aeaded6`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:09:57.178475 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d6865826-1185-4596-b6cf-b12e32cf3c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:57.252785 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1014/2024] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:09:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:09:57.477352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e77c6-8c0d-4857-8eef-eaf36894e9d1 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:57.478946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-281e77c6-8c0d-4857-8eef-eaf36894e9d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:57.492094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e77c6-8c0d-4857-8eef-eaf36894e9d1 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:57.497509 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:57.498773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:57.499010 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:57.499375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] PATH_INFO: `/v3/projects/1a4306d15f99496785653e78a488f3ea` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:57.500641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a4306d15f99496785653e78a488f3ea)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:57.503270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:57.581152 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a4306d15f99496785653e78a488f3ea', 'request_id': 'req-da184791-d0e3-4528-bc5d-3e981aba4298'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:57.588405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a4306d15f99496785653e78a488f3ea', 'request_id': 'req-da184791-d0e3-4528-bc5d-3e981aba4298'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:57.598344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a4306d15f99496785653e78a488f3ea', 'request_id': 'req-da184791-d0e3-4528-bc5d-3e981aba4298'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:57.621983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] The token cache is being invalidate because project 1a4306d15f99496785653e78a488f3ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:57.622319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:57.624126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:57.629253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1011/2025] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:57 2022] DELETE /identity/v3/projects/1a4306d15f99496785653e78a488f3ea => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:57.630544 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6865826-1185-4596-b6cf-b12e32cf3c98 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:57.644407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6865826-1185-4596-b6cf-b12e32cf3c98 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:57.657954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:57.732076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:57.732972 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:57.733250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:57.733524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] PATH_INFO: `/v3/users/1da3714c65cc468fbe717dcacc643bfa` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:57.734765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1da3714c65cc468fbe717dcacc643bfa)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:57.737232 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:57.797842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1da3714c65cc468fbe717dcacc643bfa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:57.798744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1da3714c65cc468fbe717dcacc643bfa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:57.808433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1da3714c65cc468fbe717dcacc643bfa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:57.826575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1da3714c65cc468fbe717dcacc643bfa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:57.827636 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1da3714c65cc468fbe717dcacc643bfa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:57.843742 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1015/2026] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:57 2022] DELETE /identity/v3/users/1da3714c65cc468fbe717dcacc643bfa => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:58.246999 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:58.248902 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:58.265870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da184791-d0e3-4528-bc5d-3e981aba4298 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:58.275174 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:58.351525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:58.352790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:58.352790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:58.353174 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] PATH_INFO: `/v3/projects/ca947389b8564b3a9a4f8c6d319872a0` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:58.354489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca947389b8564b3a9a4f8c6d319872a0)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:58.356456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:58.409835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca947389b8564b3a9a4f8c6d319872a0', 'request_id': 'req-2382f4b9-92a9-40fe-be69-de9998591795'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:58.416708 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca947389b8564b3a9a4f8c6d319872a0', 'request_id': 'req-2382f4b9-92a9-40fe-be69-de9998591795'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:58.422820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca947389b8564b3a9a4f8c6d319872a0', 'request_id': 'req-2382f4b9-92a9-40fe-be69-de9998591795'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:58.443301 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] The token cache is being invalidate because project ca947389b8564b3a9a4f8c6d319872a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:58.443688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:58.444463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:58.449732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1012/2027] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:58 2022] DELETE /identity/v3/projects/ca947389b8564b3a9a4f8c6d319872a0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:58.451338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:58.453116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:58.469904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e3ad-ddbb-49ba-b1c0-498699047e7a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:58.536824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:58.538011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:58.538368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:58.538716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] PATH_INFO: `/v3/users/cfa4c4f41d4a47d98a9c3af65eab2b35` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:58.540075 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa4c4f41d4a47d98a9c3af65eab2b35)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:58.542355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:58.598152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:58.611297 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa4c4f41d4a47d98a9c3af65eab2b35'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:58.612663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa4c4f41d4a47d98a9c3af65eab2b35'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:58.627154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa4c4f41d4a47d98a9c3af65eab2b35'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:58.645873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa4c4f41d4a47d98a9c3af65eab2b35'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:58.646640 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa4c4f41d4a47d98a9c3af65eab2b35'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:58.660667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1016/2028] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:09:58 2022] DELETE /identity/v3/users/cfa4c4f41d4a47d98a9c3af65eab2b35 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:09:59.100376 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:09:59.102333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:09:59.153908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382f4b9-92a9-40fe-be69-de9998591795 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:09:59.247908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:09:59.249057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:09:59.249354 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:09:59.249682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] PATH_INFO: `/v3/projects/756486e047be4714a5827ce01668ddae` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:09:59.251119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=756486e047be4714a5827ce01668ddae)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:09:59.263289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:09:59.329533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:09:59.340003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '756486e047be4714a5827ce01668ddae', 'request_id': 'req-604b5067-a254-4e98-8ca5-835c32aef471'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:59.354273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '756486e047be4714a5827ce01668ddae', 'request_id': 'req-604b5067-a254-4e98-8ca5-835c32aef471'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:59.363289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '756486e047be4714a5827ce01668ddae', 'request_id': 'req-604b5067-a254-4e98-8ca5-835c32aef471'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:59.378034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] The token cache is being invalidate because project 756486e047be4714a5827ce01668ddae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:09:59.378442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:59.378995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:09:59.380764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1013/2029] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:09:59 2022] DELETE /identity/v3/projects/756486e047be4714a5827ce01668ddae => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:02.767499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:02.770020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:02.786671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d981282-f8ef-45da-af2d-74bbceaf8d82 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:02.861765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a0410-c307-4e27-876b-f23b2323f7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:02.862671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a0410-c307-4e27-876b-f23b2323f7e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:02.862957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a0410-c307-4e27-876b-f23b2323f7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:02.863223 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a0410-c307-4e27-876b-f23b2323f7e9 admin admin] PATH_INFO: `/v3/users/64835c93d17543668a8039caeeaa393c` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:02.864492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a0410-c307-4e27-876b-f23b2323f7e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64835c93d17543668a8039caeeaa393c)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:02.866758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a0410-c307-4e27-876b-f23b2323f7e9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:02.920975 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-373a0410-c307-4e27-876b-f23b2323f7e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64835c93d17543668a8039caeeaa393c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:02.921699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-373a0410-c307-4e27-876b-f23b2323f7e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64835c93d17543668a8039caeeaa393c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:02.931873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-373a0410-c307-4e27-876b-f23b2323f7e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64835c93d17543668a8039caeeaa393c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:02.945212 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-373a0410-c307-4e27-876b-f23b2323f7e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64835c93d17543668a8039caeeaa393c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:02.945915 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-373a0410-c307-4e27-876b-f23b2323f7e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64835c93d17543668a8039caeeaa393c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:02.959516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1017/2030] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:10:02 2022] DELETE /identity/v3/users/64835c93d17543668a8039caeeaa393c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:03.148751 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:03.150319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:03.166388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b5067-a254-4e98-8ca5-835c32aef471 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:03.229456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82acf8b-87c9-405f-a310-ab5aff634a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:03.230503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82acf8b-87c9-405f-a310-ab5aff634a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:03.230788 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82acf8b-87c9-405f-a310-ab5aff634a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:03.231066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82acf8b-87c9-405f-a310-ab5aff634a87 admin admin] PATH_INFO: `/v3/projects/c5f718c1e8ae45dbb633172c0ba74c5e` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:03.232256 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82acf8b-87c9-405f-a310-ab5aff634a87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f718c1e8ae45dbb633172c0ba74c5e)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:03.234293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82acf8b-87c9-405f-a310-ab5aff634a87 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:03.288437 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b82acf8b-87c9-405f-a310-ab5aff634a87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f718c1e8ae45dbb633172c0ba74c5e', 'request_id': 'req-b82acf8b-87c9-405f-a310-ab5aff634a87'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:03.295228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b82acf8b-87c9-405f-a310-ab5aff634a87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f718c1e8ae45dbb633172c0ba74c5e', 'request_id': 'req-b82acf8b-87c9-405f-a310-ab5aff634a87'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:03.301737 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b82acf8b-87c9-405f-a310-ab5aff634a87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f718c1e8ae45dbb633172c0ba74c5e', 'request_id': 'req-b82acf8b-87c9-405f-a310-ab5aff634a87'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:03.316797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b82acf8b-87c9-405f-a310-ab5aff634a87 admin admin] The token cache is being invalidate because project c5f718c1e8ae45dbb633172c0ba74c5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:10:03.317017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b82acf8b-87c9-405f-a310-ab5aff634a87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:03.317686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b82acf8b-87c9-405f-a310-ab5aff634a87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:03.319236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1014/2031] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:10:03 2022] DELETE /identity/v3/projects/c5f718c1e8ae45dbb633172c0ba74c5e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:03.324456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024b6cc-d7f8-428d-bb1e-1a10c2e9fd18 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:03.324745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024b6cc-d7f8-428d-bb1e-1a10c2e9fd18 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:03.325027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024b6cc-d7f8-428d-bb1e-1a10c2e9fd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:03.347351 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b024b6cc-d7f8-428d-bb1e-1a10c2e9fd18 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:03.348599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b024b6cc-d7f8-428d-bb1e-1a10c2e9fd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:03.383733 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:03.420323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1018/2032] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:03.429523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383d553-3830-4ae6-8db2-0a613c61c666 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:03.429843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383d553-3830-4ae6-8db2-0a613c61c666 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:03.430121 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383d553-3830-4ae6-8db2-0a613c61c666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:03.451464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f383d553-3830-4ae6-8db2-0a613c61c666 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:03.452689 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f383d553-3830-4ae6-8db2-0a613c61c666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:03.461645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1015/2033] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:03.466110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024b6cc-d7f8-428d-bb1e-1a10c2e9fd18 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:03.480043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024b6cc-d7f8-428d-bb1e-1a10c2e9fd18 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:03.488997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229a523-0979-47d1-88b3-d87725cd8be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:03.489720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229a523-0979-47d1-88b3-d87725cd8be3 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:03.490010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229a523-0979-47d1-88b3-d87725cd8be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:03.490344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229a523-0979-47d1-88b3-d87725cd8be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:03.491091 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229a523-0979-47d1-88b3-d87725cd8be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:10:03.491402 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229a523-0979-47d1-88b3-d87725cd8be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:03.493241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229a523-0979-47d1-88b3-d87725cd8be3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:03.509142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1019/2034] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:10:03.513886 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383d553-3830-4ae6-8db2-0a613c61c666 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:03.524020 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383d553-3830-4ae6-8db2-0a613c61c666 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:03.529854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e93f20-3dcb-433b-a280-7c2fb634a1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:03.530557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93f20-3dcb-433b-a280-7c2fb634a1cd admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:03.530836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93f20-3dcb-433b-a280-7c2fb634a1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:03.531113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93f20-3dcb-433b-a280-7c2fb634a1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:03.531809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e93f20-3dcb-433b-a280-7c2fb634a1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:03.534156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e93f20-3dcb-433b-a280-7c2fb634a1cd admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:03.563579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1016/2035] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:10:03 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:03.568147 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229a523-0979-47d1-88b3-d87725cd8be3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:03.578522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229a523-0979-47d1-88b3-d87725cd8be3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:03.635667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8601a1a-1b9a-48e7-a32f-2225f58be1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:03.636145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8601a1a-1b9a-48e7-a32f-2225f58be1a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:03.636429 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8601a1a-1b9a-48e7-a32f-2225f58be1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:03.636706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8601a1a-1b9a-48e7-a32f-2225f58be1a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:03.637406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8601a1a-1b9a-48e7-a32f-2225f58be1a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:03.639440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8601a1a-1b9a-48e7-a32f-2225f58be1a5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:03.665299 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1020/2036] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:10:03 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:10:03.669651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e93f20-3dcb-433b-a280-7c2fb634a1cd admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:03.681179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e93f20-3dcb-433b-a280-7c2fb634a1cd admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:03.686242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2b483-21ca-489c-b4c9-5fbba52114a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:03.687057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2b483-21ca-489c-b4c9-5fbba52114a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:03.687407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2b483-21ca-489c-b4c9-5fbba52114a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:03.687759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2b483-21ca-489c-b4c9-5fbba52114a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:03.688568 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2b483-21ca-489c-b4c9-5fbba52114a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:03.690383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2b483-21ca-489c-b4c9-5fbba52114a8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:03.703829 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1017/2037] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:03.708272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8601a1a-1b9a-48e7-a32f-2225f58be1a5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:03.719465 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8601a1a-1b9a-48e7-a32f-2225f58be1a5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:03.725337 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ad7d5-6721-4cfe-a717-28cf26f30882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:03.726421 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ad7d5-6721-4cfe-a717-28cf26f30882 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:03.726421 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ad7d5-6721-4cfe-a717-28cf26f30882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:03.726617 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ad7d5-6721-4cfe-a717-28cf26f30882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:03.727322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ad7d5-6721-4cfe-a717-28cf26f30882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:03.728914 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ad7d5-6721-4cfe-a717-28cf26f30882 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:03.742087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1021/2038] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:03.746367 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2b483-21ca-489c-b4c9-5fbba52114a8 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:03.756591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2b483-21ca-489c-b4c9-5fbba52114a8 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:03.761242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ab21e-547d-4d05-9338-9f053517b54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:03.762179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ab21e-547d-4d05-9338-9f053517b54d admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:03.762464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ab21e-547d-4d05-9338-9f053517b54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:03.762744 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ab21e-547d-4d05-9338-9f053517b54d admin admin] PATH_INFO: `/v3/projects/cfe8cbe051a0463f984759c7e5b091db/users/eb9b38d7c3f24c97b3160de2446702f2/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:03.777681 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ab21e-547d-4d05-9338-9f053517b54d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe8cbe051a0463f984759c7e5b091db, user_id=eb9b38d7c3f24c97b3160de2446702f2, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:03.780360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ab21e-547d-4d05-9338-9f053517b54d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:03.783781 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:03.798964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1018/2039] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:03 2022] PUT /identity/v3/projects/cfe8cbe051a0463f984759c7e5b091db/users/eb9b38d7c3f24c97b3160de2446702f2/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:03.809318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f8c1b-a6d3-482b-87ae-15c305ebf616 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:03.809814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f8c1b-a6d3-482b-87ae-15c305ebf616 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:03.809907 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f8c1b-a6d3-482b-87ae-15c305ebf616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:03.839192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-836f8c1b-a6d3-482b-87ae-15c305ebf616 None None] MFA Rules not processed for user `eb9b38d7c3f24c97b3160de2446702f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:03.841882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-836f8c1b-a6d3-482b-87ae-15c305ebf616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:03.915725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1022/2040] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:03.920464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ab21e-547d-4d05-9338-9f053517b54d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:03.993119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ab21e-547d-4d05-9338-9f053517b54d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:03.999520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1852de7-e85e-41fa-8e2a-60a748cd527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:04.000359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1852de7-e85e-41fa-8e2a-60a748cd527e admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:04.000522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1852de7-e85e-41fa-8e2a-60a748cd527e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:04.000779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1852de7-e85e-41fa-8e2a-60a748cd527e admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:04.001581 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1852de7-e85e-41fa-8e2a-60a748cd527e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:04.004311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1852de7-e85e-41fa-8e2a-60a748cd527e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:04.038093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1019/2041] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:10:03 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:04.041566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f8c1b-a6d3-482b-87ae-15c305ebf616 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:04.057572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f8c1b-a6d3-482b-87ae-15c305ebf616 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:04.092937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:04.127419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b83c1-1a60-490f-b02b-5d060d8eff1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:04.128125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b83c1-1a60-490f-b02b-5d060d8eff1f admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:04.128400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b83c1-1a60-490f-b02b-5d060d8eff1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:04.128672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b83c1-1a60-490f-b02b-5d060d8eff1f admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:04.129842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b83c1-1a60-490f-b02b-5d060d8eff1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:04.131832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b83c1-1a60-490f-b02b-5d060d8eff1f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:04.160300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1023/2042] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:10:04 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:04.161588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1852de7-e85e-41fa-8e2a-60a748cd527e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:04.177304 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1852de7-e85e-41fa-8e2a-60a748cd527e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:04.182241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e269d13-f3b4-44df-8e4d-f4cead34867a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:04.183021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e269d13-f3b4-44df-8e4d-f4cead34867a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:04.183383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e269d13-f3b4-44df-8e4d-f4cead34867a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:04.183733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e269d13-f3b4-44df-8e4d-f4cead34867a admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:04.184544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e269d13-f3b4-44df-8e4d-f4cead34867a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:04.186300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e269d13-f3b4-44df-8e4d-f4cead34867a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:04.197961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1020/2043] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:04.202159 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b83c1-1a60-490f-b02b-5d060d8eff1f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:04.212624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b83c1-1a60-490f-b02b-5d060d8eff1f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:04.217467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459edec-047e-4d14-bf4d-0ab386018339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:04.218403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459edec-047e-4d14-bf4d-0ab386018339 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:04.218686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459edec-047e-4d14-bf4d-0ab386018339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:04.219308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459edec-047e-4d14-bf4d-0ab386018339 admin admin] PATH_INFO: `/v3/projects/88a262d5611649d5b5ad2e2c07e74e4f/users/bc4f928d22c547b09476b670cb23ac34/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:04.234999 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459edec-047e-4d14-bf4d-0ab386018339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88a262d5611649d5b5ad2e2c07e74e4f, user_id=bc4f928d22c547b09476b670cb23ac34, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:04.237969 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459edec-047e-4d14-bf4d-0ab386018339 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:04.251651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1024/2044] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:04 2022] PUT /identity/v3/projects/88a262d5611649d5b5ad2e2c07e74e4f/users/bc4f928d22c547b09476b670cb23ac34/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:04.256018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e269d13-f3b4-44df-8e4d-f4cead34867a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:04.266934 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e269d13-f3b4-44df-8e4d-f4cead34867a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:04.327972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1a727-0290-41aa-a4c9-a67f9341f624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:04.328660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1a727-0290-41aa-a4c9-a67f9341f624 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:04.328950 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1a727-0290-41aa-a4c9-a67f9341f624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:04.329249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1a727-0290-41aa-a4c9-a67f9341f624 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:04.330109 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1a727-0290-41aa-a4c9-a67f9341f624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:04.331724 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1a727-0290-41aa-a4c9-a67f9341f624 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:04.342093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1021/2045] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:04.347106 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459edec-047e-4d14-bf4d-0ab386018339 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:04.357593 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459edec-047e-4d14-bf4d-0ab386018339 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:04.362667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf38b9-4566-49b7-9ca3-2a03abbfdb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:04.363573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf38b9-4566-49b7-9ca3-2a03abbfdb55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:04.363858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf38b9-4566-49b7-9ca3-2a03abbfdb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:04.364137 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf38b9-4566-49b7-9ca3-2a03abbfdb55 admin admin] PATH_INFO: `/v3/projects/88a262d5611649d5b5ad2e2c07e74e4f/users/bc4f928d22c547b09476b670cb23ac34/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:04.366330 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf38b9-4566-49b7-9ca3-2a03abbfdb55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88a262d5611649d5b5ad2e2c07e74e4f, user_id=bc4f928d22c547b09476b670cb23ac34, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:04.369039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf38b9-4566-49b7-9ca3-2a03abbfdb55 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:04.379913 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1025/2046] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:04 2022] PUT /identity/v3/projects/88a262d5611649d5b5ad2e2c07e74e4f/users/bc4f928d22c547b09476b670cb23ac34/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:04.390293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7a957-2ed1-4fc2-ba6a-507c6066c99b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:04.390589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7a957-2ed1-4fc2-ba6a-507c6066c99b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:04.390871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7a957-2ed1-4fc2-ba6a-507c6066c99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:04.419369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1cb7a957-2ed1-4fc2-ba6a-507c6066c99b None None] MFA Rules not processed for user `bc4f928d22c547b09476b670cb23ac34`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:04.420669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1cb7a957-2ed1-4fc2-ba6a-507c6066c99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:04.477159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:04.495429 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1022/2047] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:04.507153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf38b9-4566-49b7-9ca3-2a03abbfdb55 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:04.508944 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-17bf38b9-4566-49b7-9ca3-2a03abbfdb55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:04.576220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf38b9-4566-49b7-9ca3-2a03abbfdb55 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:04.580001 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2af70-dfef-430c-ae15-59a5d8fb01a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:04.580663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2af70-dfef-430c-ae15-59a5d8fb01a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:04.580945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2af70-dfef-430c-ae15-59a5d8fb01a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:04.581221 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2af70-dfef-430c-ae15-59a5d8fb01a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:04.591027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2af70-dfef-430c-ae15-59a5d8fb01a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:04.592499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2af70-dfef-430c-ae15-59a5d8fb01a9 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:04.603378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1026/2048] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:04.681974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7a957-2ed1-4fc2-ba6a-507c6066c99b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:04.692305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7a957-2ed1-4fc2-ba6a-507c6066c99b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:04.696511 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0bc4f-3dc2-4656-974d-93a34905ca00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:04.697204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0bc4f-3dc2-4656-974d-93a34905ca00 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:04.697554 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0bc4f-3dc2-4656-974d-93a34905ca00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:04.697910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0bc4f-3dc2-4656-974d-93a34905ca00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:04.765327 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0bc4f-3dc2-4656-974d-93a34905ca00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:04.767225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0bc4f-3dc2-4656-974d-93a34905ca00 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:04.776532 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1023/2049] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:04 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:14.598539 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=56,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:14.772134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=32 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:15.174942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2af70-dfef-430c-ae15-59a5d8fb01a9 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:15.239072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2af70-dfef-430c-ae15-59a5d8fb01a9 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:15.243863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:15.244800 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:15.245034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:15.245312 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] PATH_INFO: `/v3/users/eb9b38d7c3f24c97b3160de2446702f2` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:15.246563 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb9b38d7c3f24c97b3160de2446702f2)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:15.248505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:15.298330 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb9b38d7c3f24c97b3160de2446702f2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:15.299001 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb9b38d7c3f24c97b3160de2446702f2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:15.312215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb9b38d7c3f24c97b3160de2446702f2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:15.324785 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb9b38d7c3f24c97b3160de2446702f2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:15.325510 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb9b38d7c3f24c97b3160de2446702f2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:15.337819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1027/2050] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:10:15 2022] DELETE /identity/v3/users/eb9b38d7c3f24c97b3160de2446702f2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:15.342438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb0668-322e-4642-9f21-20ef2f461c4e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:15.342733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb0668-322e-4642-9f21-20ef2f461c4e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:15.343016 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb0668-322e-4642-9f21-20ef2f461c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:15.360663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f0fb0668-322e-4642-9f21-20ef2f461c4e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:15.361918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f0fb0668-322e-4642-9f21-20ef2f461c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:15.422414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1024/2051] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:15.433676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:15.435263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:15.501265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c84f25-d015-486b-aeb7-da90c0a7dab4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:15.505184 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495930f-46ee-4f84-956f-8ad092fee032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:15.505856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495930f-46ee-4f84-956f-8ad092fee032 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:15.506140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495930f-46ee-4f84-956f-8ad092fee032 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:15.506413 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495930f-46ee-4f84-956f-8ad092fee032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:15.515883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495930f-46ee-4f84-956f-8ad092fee032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:15.517345 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495930f-46ee-4f84-956f-8ad092fee032 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:15.526773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1028/2052] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:15.804777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb0668-322e-4642-9f21-20ef2f461c4e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:15.806243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f0fb0668-322e-4642-9f21-20ef2f461c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:15.818489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb0668-322e-4642-9f21-20ef2f461c4e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:15.823477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:15.824392 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:15.824694 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:15.824973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] PATH_INFO: `/v3/projects/cfe8cbe051a0463f984759c7e5b091db` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:15.826236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe8cbe051a0463f984759c7e5b091db)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:15.828218 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:15.881018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe8cbe051a0463f984759c7e5b091db', 'request_id': 'req-7623901c-c1e7-4be7-a97a-05ab9b4c177b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:15.887439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe8cbe051a0463f984759c7e5b091db', 'request_id': 'req-7623901c-c1e7-4be7-a97a-05ab9b4c177b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:15.894170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe8cbe051a0463f984759c7e5b091db', 'request_id': 'req-7623901c-c1e7-4be7-a97a-05ab9b4c177b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:15.911033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] The token cache is being invalidate because project cfe8cbe051a0463f984759c7e5b091db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:10:15.911388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:15.912122 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:15.913937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1025/2053] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:10:15 2022] DELETE /identity/v3/projects/cfe8cbe051a0463f984759c7e5b091db => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:15.917979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495930f-46ee-4f84-956f-8ad092fee032 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:15.928497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495930f-46ee-4f84-956f-8ad092fee032 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:15.949524 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:15.993607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706a35c-3261-4cc3-83b3-48dc72dbbaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:15.994519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706a35c-3261-4cc3-83b3-48dc72dbbaf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:15.994798 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706a35c-3261-4cc3-83b3-48dc72dbbaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:15.995070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706a35c-3261-4cc3-83b3-48dc72dbbaf1 admin admin] PATH_INFO: `/v3/users/bc4f928d22c547b09476b670cb23ac34` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:15.996453 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706a35c-3261-4cc3-83b3-48dc72dbbaf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc4f928d22c547b09476b670cb23ac34)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:15.999026 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706a35c-3261-4cc3-83b3-48dc72dbbaf1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:16.052322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6706a35c-3261-4cc3-83b3-48dc72dbbaf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc4f928d22c547b09476b670cb23ac34'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:16.053108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6706a35c-3261-4cc3-83b3-48dc72dbbaf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc4f928d22c547b09476b670cb23ac34'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:16.063036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6706a35c-3261-4cc3-83b3-48dc72dbbaf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc4f928d22c547b09476b670cb23ac34'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:16.075049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6706a35c-3261-4cc3-83b3-48dc72dbbaf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc4f928d22c547b09476b670cb23ac34'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:16.075716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6706a35c-3261-4cc3-83b3-48dc72dbbaf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc4f928d22c547b09476b670cb23ac34'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:16.087941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1029/2054] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:10:15 2022] DELETE /identity/v3/users/bc4f928d22c547b09476b670cb23ac34 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:16.377606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:16.379181 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:16.392160 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623901c-c1e7-4be7-a97a-05ab9b4c177b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:16.465490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99fde6-c90a-4471-a338-12bdbafe4ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:16.466681 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99fde6-c90a-4471-a338-12bdbafe4ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:16.466963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99fde6-c90a-4471-a338-12bdbafe4ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:16.467307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99fde6-c90a-4471-a338-12bdbafe4ad4 admin admin] PATH_INFO: `/v3/projects/88a262d5611649d5b5ad2e2c07e74e4f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:16.468526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99fde6-c90a-4471-a338-12bdbafe4ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88a262d5611649d5b5ad2e2c07e74e4f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:16.470895 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99fde6-c90a-4471-a338-12bdbafe4ad4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:16.506157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:16.523453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5a99fde6-c90a-4471-a338-12bdbafe4ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88a262d5611649d5b5ad2e2c07e74e4f', 'request_id': 'req-5a99fde6-c90a-4471-a338-12bdbafe4ad4'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:16.529818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5a99fde6-c90a-4471-a338-12bdbafe4ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88a262d5611649d5b5ad2e2c07e74e4f', 'request_id': 'req-5a99fde6-c90a-4471-a338-12bdbafe4ad4'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:16.535590 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5a99fde6-c90a-4471-a338-12bdbafe4ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88a262d5611649d5b5ad2e2c07e74e4f', 'request_id': 'req-5a99fde6-c90a-4471-a338-12bdbafe4ad4'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:16.548880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5a99fde6-c90a-4471-a338-12bdbafe4ad4 admin admin] The token cache is being invalidate because project 88a262d5611649d5b5ad2e2c07e74e4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:10:16.549169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5a99fde6-c90a-4471-a338-12bdbafe4ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:16.549819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5a99fde6-c90a-4471-a338-12bdbafe4ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:16.551451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1026/2055] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:10:16 2022] DELETE /identity/v3/projects/88a262d5611649d5b5ad2e2c07e74e4f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:16.556640 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10df3d-8258-4405-9aee-b644c096a643 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:16.556935 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10df3d-8258-4405-9aee-b644c096a643 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:16.557213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10df3d-8258-4405-9aee-b644c096a643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:16.574876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ac10df3d-8258-4405-9aee-b644c096a643 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:16.576093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ac10df3d-8258-4405-9aee-b644c096a643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:16.641769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1030/2056] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:16.650429 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d774136-7cee-4f1b-a83a-a1f7c8af1ca3 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:16.650678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d774136-7cee-4f1b-a83a-a1f7c8af1ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:16.650956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d774136-7cee-4f1b-a83a-a1f7c8af1ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:16.671709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0d774136-7cee-4f1b-a83a-a1f7c8af1ca3 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:16.673047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0d774136-7cee-4f1b-a83a-a1f7c8af1ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:16.681908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1027/2057] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:16.686502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10df3d-8258-4405-9aee-b644c096a643 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:16.697346 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10df3d-8258-4405-9aee-b644c096a643 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:16.702326 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4e4ec-979f-45ec-8cb4-9cfc8965294a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:16.702986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4e4ec-979f-45ec-8cb4-9cfc8965294a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:16.703286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4e4ec-979f-45ec-8cb4-9cfc8965294a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:16.703564 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4e4ec-979f-45ec-8cb4-9cfc8965294a admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:16.704303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4e4ec-979f-45ec-8cb4-9cfc8965294a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:10:16.704609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4e4ec-979f-45ec-8cb4-9cfc8965294a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:16.706276 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4e4ec-979f-45ec-8cb4-9cfc8965294a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:16.718663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1031/2058] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:10:16.723628 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d774136-7cee-4f1b-a83a-a1f7c8af1ca3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:16.735284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d774136-7cee-4f1b-a83a-a1f7c8af1ca3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:16.739982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda9168-075f-4ab0-9a67-6b4f007e907e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:16.740640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda9168-075f-4ab0-9a67-6b4f007e907e admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:16.740919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda9168-075f-4ab0-9a67-6b4f007e907e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:16.741184 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda9168-075f-4ab0-9a67-6b4f007e907e admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:16.741956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda9168-075f-4ab0-9a67-6b4f007e907e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:16.743908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda9168-075f-4ab0-9a67-6b4f007e907e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:16.772964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1028/2059] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:10:16 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:16.777302 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4e4ec-979f-45ec-8cb4-9cfc8965294a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:16.787939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4e4ec-979f-45ec-8cb4-9cfc8965294a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:16.819148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:16.861527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da91c0-e955-451b-9fbb-f588c6bdb8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:16.862320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da91c0-e955-451b-9fbb-f588c6bdb8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:16.862600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da91c0-e955-451b-9fbb-f588c6bdb8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:16.862874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da91c0-e955-451b-9fbb-f588c6bdb8c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:16.863573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da91c0-e955-451b-9fbb-f588c6bdb8c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:16.865562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da91c0-e955-451b-9fbb-f588c6bdb8c0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:16.891187 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1032/2060] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:10:16 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:10:16.895233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda9168-075f-4ab0-9a67-6b4f007e907e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:16.909015 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda9168-075f-4ab0-9a67-6b4f007e907e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:16.915878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c7024-f33e-492b-8663-aee09c87de84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:16.916661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c7024-f33e-492b-8663-aee09c87de84 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:16.917125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c7024-f33e-492b-8663-aee09c87de84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:16.917409 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c7024-f33e-492b-8663-aee09c87de84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:16.918370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c7024-f33e-492b-8663-aee09c87de84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:16.920300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c7024-f33e-492b-8663-aee09c87de84 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:16.931907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1029/2061] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:16.936411 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da91c0-e955-451b-9fbb-f588c6bdb8c0 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:16.946939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da91c0-e955-451b-9fbb-f588c6bdb8c0 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:16.951612 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f2e13-05f6-4941-9fbc-31c8355919cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:16.952320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f2e13-05f6-4941-9fbc-31c8355919cd admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:16.952604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f2e13-05f6-4941-9fbc-31c8355919cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:16.952941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f2e13-05f6-4941-9fbc-31c8355919cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:16.953712 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f2e13-05f6-4941-9fbc-31c8355919cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:16.955372 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f2e13-05f6-4941-9fbc-31c8355919cd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:16.966687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1033/2062] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:16.971170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c7024-f33e-492b-8663-aee09c87de84 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:16.981672 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c7024-f33e-492b-8663-aee09c87de84 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:16.986418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc05637-5b65-4f56-a096-31271aabbb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:16.987244 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc05637-5b65-4f56-a096-31271aabbb03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:16.987528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc05637-5b65-4f56-a096-31271aabbb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:16.987804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc05637-5b65-4f56-a096-31271aabbb03 admin admin] PATH_INFO: `/v3/projects/dd0427d84a06469e9f67b2f032ff43e5/users/2f71f8ffb78d4ca5a0680c28cf7ac62f/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:17.005066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc05637-5b65-4f56-a096-31271aabbb03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0427d84a06469e9f67b2f032ff43e5, user_id=2f71f8ffb78d4ca5a0680c28cf7ac62f, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:17.008095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc05637-5b65-4f56-a096-31271aabbb03 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:17.022313 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1030/2063] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:16 2022] PUT /identity/v3/projects/dd0427d84a06469e9f67b2f032ff43e5/users/2f71f8ffb78d4ca5a0680c28cf7ac62f/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:10:17.032871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6821091-d974-413b-a0e3-aa8704a18d5d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:17.033188 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6821091-d974-413b-a0e3-aa8704a18d5d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:17.033492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6821091-d974-413b-a0e3-aa8704a18d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:17.061155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b6821091-d974-413b-a0e3-aa8704a18d5d None None] MFA Rules not processed for user `2f71f8ffb78d4ca5a0680c28cf7ac62f`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:17.062332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b6821091-d974-413b-a0e3-aa8704a18d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:17.134548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1034/2064] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:17.139578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc05637-5b65-4f56-a096-31271aabbb03 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:17.212918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc05637-5b65-4f56-a096-31271aabbb03 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:17.217698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303c3a9-fa92-485b-8e07-b5ca68b3cf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:17.218362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303c3a9-fa92-485b-8e07-b5ca68b3cf94 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:17.218638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303c3a9-fa92-485b-8e07-b5ca68b3cf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:17.219260 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303c3a9-fa92-485b-8e07-b5ca68b3cf94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:17.219951 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303c3a9-fa92-485b-8e07-b5ca68b3cf94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:17.221928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303c3a9-fa92-485b-8e07-b5ca68b3cf94 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:17.249484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1031/2065] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:10:17 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:17.253705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6821091-d974-413b-a0e3-aa8704a18d5d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:17.265053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6821091-d974-413b-a0e3-aa8704a18d5d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:17.334883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd57da-d8c2-4b29-8e20-8cda54deab65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:17.335641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd57da-d8c2-4b29-8e20-8cda54deab65 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:17.335949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd57da-d8c2-4b29-8e20-8cda54deab65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:17.336245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd57da-d8c2-4b29-8e20-8cda54deab65 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:17.337022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd57da-d8c2-4b29-8e20-8cda54deab65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:17.339193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd57da-d8c2-4b29-8e20-8cda54deab65 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:17.364696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1035/2066] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:10:17 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:17.366179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303c3a9-fa92-485b-8e07-b5ca68b3cf94 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:17.376714 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303c3a9-fa92-485b-8e07-b5ca68b3cf94 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:17.381776 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee41d0f-6085-41e1-9a15-12b3c686eec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:17.382543 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee41d0f-6085-41e1-9a15-12b3c686eec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:17.382833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee41d0f-6085-41e1-9a15-12b3c686eec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:17.383134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee41d0f-6085-41e1-9a15-12b3c686eec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:17.383869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee41d0f-6085-41e1-9a15-12b3c686eec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:17.385544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee41d0f-6085-41e1-9a15-12b3c686eec2 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:17.397273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1032/2067] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:17.402248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd57da-d8c2-4b29-8e20-8cda54deab65 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:17.418123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd57da-d8c2-4b29-8e20-8cda54deab65 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:17.423795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ceefb0-c1a2-40fa-9db0-a79ae0cc97cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:17.424499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceefb0-c1a2-40fa-9db0-a79ae0cc97cc admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:17.424783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceefb0-c1a2-40fa-9db0-a79ae0cc97cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:17.425208 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceefb0-c1a2-40fa-9db0-a79ae0cc97cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:17.426117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ceefb0-c1a2-40fa-9db0-a79ae0cc97cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:17.427928 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ceefb0-c1a2-40fa-9db0-a79ae0cc97cc admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:17.441285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1036/2068] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:10:17.445893 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee41d0f-6085-41e1-9a15-12b3c686eec2 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:17.456958 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee41d0f-6085-41e1-9a15-12b3c686eec2 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:17.463245 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d208e4-d631-4a22-99ed-ee77c4a5ebd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:17.464295 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d208e4-d631-4a22-99ed-ee77c4a5ebd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:17.464576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d208e4-d631-4a22-99ed-ee77c4a5ebd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:17.464847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d208e4-d631-4a22-99ed-ee77c4a5ebd5 admin admin] PATH_INFO: `/v3/projects/75f6261b90f54ee3b43dfa3364416528/users/723c7a01be6f46a08bb4bd1a5fec605d/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:17.481363 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d208e4-d631-4a22-99ed-ee77c4a5ebd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f6261b90f54ee3b43dfa3364416528, user_id=723c7a01be6f46a08bb4bd1a5fec605d, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:17.484393 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d208e4-d631-4a22-99ed-ee77c4a5ebd5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:17.496402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:17.503526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1033/2069] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:17 2022] PUT /identity/v3/projects/75f6261b90f54ee3b43dfa3364416528/users/723c7a01be6f46a08bb4bd1a5fec605d/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:17.515925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff5122-aec5-4b79-99c0-57f5bc0a2e07 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:17.516215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff5122-aec5-4b79-99c0-57f5bc0a2e07 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:17.516552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff5122-aec5-4b79-99c0-57f5bc0a2e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:17.544952 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-12ff5122-aec5-4b79-99c0-57f5bc0a2e07 None None] MFA Rules not processed for user `723c7a01be6f46a08bb4bd1a5fec605d`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:17.546219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-12ff5122-aec5-4b79-99c0-57f5bc0a2e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:17.550364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:17.624409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1037/2070] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:10:17.629419 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d208e4-d631-4a22-99ed-ee77c4a5ebd5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:17.724643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d208e4-d631-4a22-99ed-ee77c4a5ebd5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:17.730928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36528318-657d-428a-83a0-147205d09b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:17.731685 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36528318-657d-428a-83a0-147205d09b5f admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:17.732030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36528318-657d-428a-83a0-147205d09b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:17.732345 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36528318-657d-428a-83a0-147205d09b5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:17.733099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36528318-657d-428a-83a0-147205d09b5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:17.737996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36528318-657d-428a-83a0-147205d09b5f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:17.772207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1034/2071] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:10:17 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:17.776774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff5122-aec5-4b79-99c0-57f5bc0a2e07 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:17.789794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff5122-aec5-4b79-99c0-57f5bc0a2e07 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:17.858542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b71bce-aed8-47ca-907e-6e7e9693cb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:17.859525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b71bce-aed8-47ca-907e-6e7e9693cb71 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:17.859873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b71bce-aed8-47ca-907e-6e7e9693cb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:17.860361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b71bce-aed8-47ca-907e-6e7e9693cb71 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:17.862046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b71bce-aed8-47ca-907e-6e7e9693cb71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:17.864105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b71bce-aed8-47ca-907e-6e7e9693cb71 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:17.892416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1038/2072] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:10:17 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:17.896492 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36528318-657d-428a-83a0-147205d09b5f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:17.907918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36528318-657d-428a-83a0-147205d09b5f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:17.914571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16930553-c447-45e4-8e7a-19cbdba6233c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:17.915650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16930553-c447-45e4-8e7a-19cbdba6233c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:17.915945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16930553-c447-45e4-8e7a-19cbdba6233c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:17.916210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16930553-c447-45e4-8e7a-19cbdba6233c admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:17.917290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16930553-c447-45e4-8e7a-19cbdba6233c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:17.919435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16930553-c447-45e4-8e7a-19cbdba6233c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:17.933361 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1035/2073] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:17.941115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b71bce-aed8-47ca-907e-6e7e9693cb71 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:17.952670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b71bce-aed8-47ca-907e-6e7e9693cb71 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:17.958675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84211d5b-0038-4f73-a616-07604a2b4c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:17.959646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84211d5b-0038-4f73-a616-07604a2b4c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:17.959892 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84211d5b-0038-4f73-a616-07604a2b4c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:17.960262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84211d5b-0038-4f73-a616-07604a2b4c19 admin admin] PATH_INFO: `/v3/projects/c9ecc167515c4ce88616bfbef7461985/users/e6c2a3a1919145759718c8f7f1212aa0/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:17.982693 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84211d5b-0038-4f73-a616-07604a2b4c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ecc167515c4ce88616bfbef7461985, user_id=e6c2a3a1919145759718c8f7f1212aa0, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:17.985552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84211d5b-0038-4f73-a616-07604a2b4c19 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:18.002478 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1039/2074] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:17 2022] PUT /identity/v3/projects/c9ecc167515c4ce88616bfbef7461985/users/e6c2a3a1919145759718c8f7f1212aa0/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:18.006533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16930553-c447-45e4-8e7a-19cbdba6233c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:18.022888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16930553-c447-45e4-8e7a-19cbdba6233c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:18.086307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e450160-1024-4702-ad80-b89d105f6365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:18.087457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e450160-1024-4702-ad80-b89d105f6365 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:18.087755 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e450160-1024-4702-ad80-b89d105f6365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:18.088045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e450160-1024-4702-ad80-b89d105f6365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:18.088859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e450160-1024-4702-ad80-b89d105f6365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:18.090513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e450160-1024-4702-ad80-b89d105f6365 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:18.103984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1036/2075] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:18.107064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84211d5b-0038-4f73-a616-07604a2b4c19 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:18.119088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84211d5b-0038-4f73-a616-07604a2b4c19 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:18.129526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9c1f6-6f83-470f-979c-982cc679851d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:18.131011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9c1f6-6f83-470f-979c-982cc679851d admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:18.131011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9c1f6-6f83-470f-979c-982cc679851d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:18.131285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9c1f6-6f83-470f-979c-982cc679851d admin admin] PATH_INFO: `/v3/projects/c9ecc167515c4ce88616bfbef7461985/users/e6c2a3a1919145759718c8f7f1212aa0/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:18.133863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9c1f6-6f83-470f-979c-982cc679851d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ecc167515c4ce88616bfbef7461985, user_id=e6c2a3a1919145759718c8f7f1212aa0, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:18.136837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9c1f6-6f83-470f-979c-982cc679851d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:18.155772 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1040/2076] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:18 2022] PUT /identity/v3/projects/c9ecc167515c4ce88616bfbef7461985/users/e6c2a3a1919145759718c8f7f1212aa0/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:10:18.166524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81da6aa-69f8-4a17-85a5-c4f858a88817 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:18.166853 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81da6aa-69f8-4a17-85a5-c4f858a88817 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:18.167135 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81da6aa-69f8-4a17-85a5-c4f858a88817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:18.202335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e81da6aa-69f8-4a17-85a5-c4f858a88817 None None] MFA Rules not processed for user `e6c2a3a1919145759718c8f7f1212aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:18.203822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e81da6aa-69f8-4a17-85a5-c4f858a88817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:18.290084 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:18.302563 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1037/2077] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:18.322134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9c1f6-6f83-470f-979c-982cc679851d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:18.324670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0cb9c1f6-6f83-470f-979c-982cc679851d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:18.398932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9c1f6-6f83-470f-979c-982cc679851d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:18.403754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfad3d9-18a9-4d4c-ac39-642dc5d08736 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:18.404406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfad3d9-18a9-4d4c-ac39-642dc5d08736 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:18.404686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfad3d9-18a9-4d4c-ac39-642dc5d08736 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:18.404958 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfad3d9-18a9-4d4c-ac39-642dc5d08736 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:18.414945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfad3d9-18a9-4d4c-ac39-642dc5d08736 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:18.416401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfad3d9-18a9-4d4c-ac39-642dc5d08736 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:18.421812 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:18.427186 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1041/2078] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:18.950420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81da6aa-69f8-4a17-85a5-c4f858a88817 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:18.951915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e81da6aa-69f8-4a17-85a5-c4f858a88817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:18.964728 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81da6aa-69f8-4a17-85a5-c4f858a88817 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:18.969583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca259b53-2b24-4e84-ba86-4e1f034efae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:18.970299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca259b53-2b24-4e84-ba86-4e1f034efae4 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:18.970585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca259b53-2b24-4e84-ba86-4e1f034efae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:18.970864 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca259b53-2b24-4e84-ba86-4e1f034efae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:19.031957 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca259b53-2b24-4e84-ba86-4e1f034efae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:19.033418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca259b53-2b24-4e84-ba86-4e1f034efae4 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:19.043078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1038/2079] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:19.191327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfad3d9-18a9-4d4c-ac39-642dc5d08736 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:19.193216 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ebfad3d9-18a9-4d4c-ac39-642dc5d08736 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:19.271015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfad3d9-18a9-4d4c-ac39-642dc5d08736 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:19.277093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:19.278163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:19.278531 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:19.278934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] PATH_INFO: `/v3/users/1e873cc6aea340939f142af452be6ae3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:19.280421 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e873cc6aea340939f142af452be6ae3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:19.283265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:19.370443 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e873cc6aea340939f142af452be6ae3'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:19.371129 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e873cc6aea340939f142af452be6ae3'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:19.380824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e873cc6aea340939f142af452be6ae3'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:19.394892 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e873cc6aea340939f142af452be6ae3'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:19.395652 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e873cc6aea340939f142af452be6ae3'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:19.410652 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1042/2080] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:10:19 2022] DELETE /identity/v3/users/1e873cc6aea340939f142af452be6ae3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:19.415284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbae6cf-312c-43d6-a2a3-e6347197b538 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:19.415663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbae6cf-312c-43d6-a2a3-e6347197b538 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:19.415966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbae6cf-312c-43d6-a2a3-e6347197b538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:19.434942 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9fbae6cf-312c-43d6-a2a3-e6347197b538 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:19.436206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9fbae6cf-312c-43d6-a2a3-e6347197b538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:19.501478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1039/2081] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:19.511604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:19.513549 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:19.572459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aab4ef-e98a-4e84-b693-6ea2b4b890f0 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:19.576042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13f260-47e4-4e79-ae80-9d25bea7dbd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:19.576690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13f260-47e4-4e79-ae80-9d25bea7dbd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:19.576971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13f260-47e4-4e79-ae80-9d25bea7dbd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:19.577244 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13f260-47e4-4e79-ae80-9d25bea7dbd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:19.588199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13f260-47e4-4e79-ae80-9d25bea7dbd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:19.589758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13f260-47e4-4e79-ae80-9d25bea7dbd8 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:19.601456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1043/2082] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:19.904613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbae6cf-312c-43d6-a2a3-e6347197b538 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:19.907511 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9fbae6cf-312c-43d6-a2a3-e6347197b538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:19.924489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbae6cf-312c-43d6-a2a3-e6347197b538 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:19.938766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe2440-5777-41b2-b952-ed30f21a3e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:19.939701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe2440-5777-41b2-b952-ed30f21a3e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:19.939980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe2440-5777-41b2-b952-ed30f21a3e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:19.940313 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe2440-5777-41b2-b952-ed30f21a3e54 admin admin] PATH_INFO: `/v3/projects/846253ae8e114af4a874751ca1f70936` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:19.941499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe2440-5777-41b2-b952-ed30f21a3e54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=846253ae8e114af4a874751ca1f70936)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:19.943464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe2440-5777-41b2-b952-ed30f21a3e54 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:20.005611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-69fe2440-5777-41b2-b952-ed30f21a3e54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846253ae8e114af4a874751ca1f70936', 'request_id': 'req-69fe2440-5777-41b2-b952-ed30f21a3e54'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:20.012726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-69fe2440-5777-41b2-b952-ed30f21a3e54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846253ae8e114af4a874751ca1f70936', 'request_id': 'req-69fe2440-5777-41b2-b952-ed30f21a3e54'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:20.018810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-69fe2440-5777-41b2-b952-ed30f21a3e54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846253ae8e114af4a874751ca1f70936', 'request_id': 'req-69fe2440-5777-41b2-b952-ed30f21a3e54'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:20.035503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-69fe2440-5777-41b2-b952-ed30f21a3e54 admin admin] The token cache is being invalidate because project 846253ae8e114af4a874751ca1f70936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:10:20.038600 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-69fe2440-5777-41b2-b952-ed30f21a3e54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:20.039663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-69fe2440-5777-41b2-b952-ed30f21a3e54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:20.041300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1040/2083] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:10:19 2022] DELETE /identity/v3/projects/846253ae8e114af4a874751ca1f70936 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:20.046309 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947087d-f031-4f69-ae3c-0009441a160c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:20.046677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947087d-f031-4f69-ae3c-0009441a160c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:20.047031 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947087d-f031-4f69-ae3c-0009441a160c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:20.066663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a947087d-f031-4f69-ae3c-0009441a160c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:20.067878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a947087d-f031-4f69-ae3c-0009441a160c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:20.087804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:20.154725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1044/2084] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:20.162772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ac93b-8588-401b-bb4b-554966b39981 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:20.163077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ac93b-8588-401b-bb4b-554966b39981 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:20.163231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ac93b-8588-401b-bb4b-554966b39981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:20.183583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9f7ac93b-8588-401b-bb4b-554966b39981 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:20.184814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9f7ac93b-8588-401b-bb4b-554966b39981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:20.196105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1041/2085] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:20.200293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947087d-f031-4f69-ae3c-0009441a160c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:20.211070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947087d-f031-4f69-ae3c-0009441a160c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:20.216315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa3cb8-29b5-4686-ad5d-17a031cb98ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:20.216994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa3cb8-29b5-4686-ad5d-17a031cb98ad admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:20.217435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa3cb8-29b5-4686-ad5d-17a031cb98ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:20.217904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa3cb8-29b5-4686-ad5d-17a031cb98ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:20.219007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa3cb8-29b5-4686-ad5d-17a031cb98ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:10:20.219292 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa3cb8-29b5-4686-ad5d-17a031cb98ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:20.221312 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa3cb8-29b5-4686-ad5d-17a031cb98ad admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:20.234459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1045/2086] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:10:20.238939 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ac93b-8588-401b-bb4b-554966b39981 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:20.248808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ac93b-8588-401b-bb4b-554966b39981 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:20.253562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be3dc3-e199-4a9a-ad48-5aba017ef56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:20.254368 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be3dc3-e199-4a9a-ad48-5aba017ef56f admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:20.254650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be3dc3-e199-4a9a-ad48-5aba017ef56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:20.254923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be3dc3-e199-4a9a-ad48-5aba017ef56f admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:20.255611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be3dc3-e199-4a9a-ad48-5aba017ef56f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:20.257805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be3dc3-e199-4a9a-ad48-5aba017ef56f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:20.266462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:20.287718 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1042/2087] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:10:20 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:10:20.292248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa3cb8-29b5-4686-ad5d-17a031cb98ad admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:20.302886 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa3cb8-29b5-4686-ad5d-17a031cb98ad admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:20.366205 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e591a95-3a0c-4488-9dc9-ea84bb592026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:20.367128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e591a95-3a0c-4488-9dc9-ea84bb592026 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:20.367273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e591a95-3a0c-4488-9dc9-ea84bb592026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:20.367518 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e591a95-3a0c-4488-9dc9-ea84bb592026 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:20.368331 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e591a95-3a0c-4488-9dc9-ea84bb592026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:20.370394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e591a95-3a0c-4488-9dc9-ea84bb592026 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:20.396299 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1046/2088] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:10:20 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:10:20.397089 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be3dc3-e199-4a9a-ad48-5aba017ef56f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:20.408432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be3dc3-e199-4a9a-ad48-5aba017ef56f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:20.413632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7a932-a20b-41a5-abcb-6f95c464382f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:20.414510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7a932-a20b-41a5-abcb-6f95c464382f admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:20.414878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7a932-a20b-41a5-abcb-6f95c464382f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:20.415221 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7a932-a20b-41a5-abcb-6f95c464382f admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:20.416081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7a932-a20b-41a5-abcb-6f95c464382f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:20.417782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7a932-a20b-41a5-abcb-6f95c464382f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:20.432053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1043/2089] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:20.433680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e591a95-3a0c-4488-9dc9-ea84bb592026 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:20.444589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e591a95-3a0c-4488-9dc9-ea84bb592026 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:20.449993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3037d-309f-4c2c-8b93-2d7b7e8a1cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:20.450709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3037d-309f-4c2c-8b93-2d7b7e8a1cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:20.451022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3037d-309f-4c2c-8b93-2d7b7e8a1cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:20.451471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3037d-309f-4c2c-8b93-2d7b7e8a1cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:20.452563 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3037d-309f-4c2c-8b93-2d7b7e8a1cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:20.454492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3037d-309f-4c2c-8b93-2d7b7e8a1cf0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:20.465842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1047/2090] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:20.470999 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7a932-a20b-41a5-abcb-6f95c464382f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:20.482069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7a932-a20b-41a5-abcb-6f95c464382f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:20.487678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f502d-4ba8-49ba-b9ad-081204ecfec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:20.488811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f502d-4ba8-49ba-b9ad-081204ecfec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:20.489263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f502d-4ba8-49ba-b9ad-081204ecfec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:20.489759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f502d-4ba8-49ba-b9ad-081204ecfec8 admin admin] PATH_INFO: `/v3/projects/f368d911b81a44dea97778601291db98/users/e38d9de21fe744bcb289f223d22076da/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:20.508241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f502d-4ba8-49ba-b9ad-081204ecfec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f368d911b81a44dea97778601291db98, user_id=e38d9de21fe744bcb289f223d22076da, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:20.513010 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f502d-4ba8-49ba-b9ad-081204ecfec8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:20.534544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1044/2091] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:20 2022] PUT /identity/v3/projects/f368d911b81a44dea97778601291db98/users/e38d9de21fe744bcb289f223d22076da/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:10:20.546640 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b89993-213f-4d05-83c7-91f77c431cdb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:20.546930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b89993-213f-4d05-83c7-91f77c431cdb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:20.547581 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b89993-213f-4d05-83c7-91f77c431cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:20.581943 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d9b89993-213f-4d05-83c7-91f77c431cdb None None] MFA Rules not processed for user `e38d9de21fe744bcb289f223d22076da`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:20.591250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d9b89993-213f-4d05-83c7-91f77c431cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:20.650603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1048/2092] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:20.661690 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f502d-4ba8-49ba-b9ad-081204ecfec8 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:20.663598 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fc1f502d-4ba8-49ba-b9ad-081204ecfec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:20.729516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f502d-4ba8-49ba-b9ad-081204ecfec8 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:20.733269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba435b-7de3-4173-9cf7-9d40b235f72f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:20.734199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba435b-7de3-4173-9cf7-9d40b235f72f service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:20.734555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba435b-7de3-4173-9cf7-9d40b235f72f service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:20.734889 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba435b-7de3-4173-9cf7-9d40b235f72f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:20.744576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba435b-7de3-4173-9cf7-9d40b235f72f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:20.746079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba435b-7de3-4173-9cf7-9d40b235f72f service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:20.755250 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1045/2093] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:24.311544 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d73ae-8c4b-40ec-a77e-fc71ad853a96 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:24.312080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d73ae-8c4b-40ec-a77e-fc71ad853a96 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:24.312315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d73ae-8c4b-40ec-a77e-fc71ad853a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:24.334917 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f50d73ae-8c4b-40ec-a77e-fc71ad853a96 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:24.336556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f50d73ae-8c4b-40ec-a77e-fc71ad853a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:24.352449 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:24.415434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1049/2094] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:10:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:24.463358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba435b-7de3-4173-9cf7-9d40b235f72f service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:24.464847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-26ba435b-7de3-4173-9cf7-9d40b235f72f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:24.536942 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba435b-7de3-4173-9cf7-9d40b235f72f service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:24.542551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab170f-4005-4102-8954-bfcd9fa48b6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:24.543236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab170f-4005-4102-8954-bfcd9fa48b6a service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:24.543524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab170f-4005-4102-8954-bfcd9fa48b6a service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:24.543801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab170f-4005-4102-8954-bfcd9fa48b6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:24.552439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab170f-4005-4102-8954-bfcd9fa48b6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:24.554082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab170f-4005-4102-8954-bfcd9fa48b6a service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:24.562775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1046/2095] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:10:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 10:10:28.465333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d73ae-8c4b-40ec-a77e-fc71ad853a96 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:28.466886 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f50d73ae-8c4b-40ec-a77e-fc71ad853a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:28.481056 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d73ae-8c4b-40ec-a77e-fc71ad853a96 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:28.484947 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6323f-cef0-490a-8f59-94144cfa9e85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:28.485811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6323f-cef0-490a-8f59-94144cfa9e85 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:28.486049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6323f-cef0-490a-8f59-94144cfa9e85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:28.486319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6323f-cef0-490a-8f59-94144cfa9e85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:28.488436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-bff6323f-cef0-490a-8f59-94144cfa9e85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:28.584177 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6323f-cef0-490a-8f59-94144cfa9e85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:28.586258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6323f-cef0-490a-8f59-94144cfa9e85 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:28.598175 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1050/2096] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:33.264990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab170f-4005-4102-8954-bfcd9fa48b6a service swift] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:33.267280 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d5ab170f-4005-4102-8954-bfcd9fa48b6a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:33.322207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:33.360133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab170f-4005-4102-8954-bfcd9fa48b6a service swift] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:33.366073 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:33.367126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:33.367412 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:33.367690 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] PATH_INFO: `/v3/users/2f71f8ffb78d4ca5a0680c28cf7ac62f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:33.369726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f71f8ffb78d4ca5a0680c28cf7ac62f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:33.371949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:33.435740 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f71f8ffb78d4ca5a0680c28cf7ac62f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:33.437665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f71f8ffb78d4ca5a0680c28cf7ac62f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:33.448921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f71f8ffb78d4ca5a0680c28cf7ac62f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:33.464163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f71f8ffb78d4ca5a0680c28cf7ac62f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:33.464915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f71f8ffb78d4ca5a0680c28cf7ac62f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:33.483402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1047/2097] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:10:33 2022] DELETE /identity/v3/users/2f71f8ffb78d4ca5a0680c28cf7ac62f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:10:33.487959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1e208-1097-470e-a3e3-1ee1e0f40f86 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:33.488526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1e208-1097-470e-a3e3-1ee1e0f40f86 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:33.488526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1e208-1097-470e-a3e3-1ee1e0f40f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:33.506412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-aab1e208-1097-470e-a3e3-1ee1e0f40f86 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:33.507613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-aab1e208-1097-470e-a3e3-1ee1e0f40f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:33.576898 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1051/2098] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:33.587059 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:33.588505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:33.663183 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab93d0c-64b0-4261-8522-dcc48aba7917 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:33.668409 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cb65c-6083-4554-894a-c5a7db33ec62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:33.669163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cb65c-6083-4554-894a-c5a7db33ec62 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:33.670397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cb65c-6083-4554-894a-c5a7db33ec62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:33.670786 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cb65c-6083-4554-894a-c5a7db33ec62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:33.683365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cb65c-6083-4554-894a-c5a7db33ec62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:33.685868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cb65c-6083-4554-894a-c5a7db33ec62 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:33.710391 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1048/2099] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:34.167323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1e208-1097-470e-a3e3-1ee1e0f40f86 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:34.169343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-aab1e208-1097-470e-a3e3-1ee1e0f40f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:34.182291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1e208-1097-470e-a3e3-1ee1e0f40f86 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:34.187724 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:34.190601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:34.190899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:34.191185 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] PATH_INFO: `/v3/projects/dd0427d84a06469e9f67b2f032ff43e5` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:34.192706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd0427d84a06469e9f67b2f032ff43e5)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:34.194849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:34.271146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd0427d84a06469e9f67b2f032ff43e5', 'request_id': 'req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:34.276303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:34.277878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd0427d84a06469e9f67b2f032ff43e5', 'request_id': 'req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:34.286712 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd0427d84a06469e9f67b2f032ff43e5', 'request_id': 'req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:34.302712 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] The token cache is being invalidate because project dd0427d84a06469e9f67b2f032ff43e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:10:34.303060 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:34.303897 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:34.305657 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1052/2100] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:10:34 2022] DELETE /identity/v3/projects/dd0427d84a06469e9f67b2f032ff43e5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:34.309679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cb65c-6083-4554-894a-c5a7db33ec62 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:34.322169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cb65c-6083-4554-894a-c5a7db33ec62 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:34.385799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:34.386705 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:34.386987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:34.387268 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] PATH_INFO: `/v3/users/723c7a01be6f46a08bb4bd1a5fec605d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:34.388554 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=723c7a01be6f46a08bb4bd1a5fec605d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:34.390536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:34.403284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:34.444654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '723c7a01be6f46a08bb4bd1a5fec605d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:34.445475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '723c7a01be6f46a08bb4bd1a5fec605d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:34.454501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '723c7a01be6f46a08bb4bd1a5fec605d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:34.468923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '723c7a01be6f46a08bb4bd1a5fec605d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:34.469719 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '723c7a01be6f46a08bb4bd1a5fec605d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:34.485388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1049/2101] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:10:34 2022] DELETE /identity/v3/users/723c7a01be6f46a08bb4bd1a5fec605d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:34.824024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:34.825542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:34.862885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0e11a-fe86-4aa0-bf6b-782727a53b9c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:34.984207 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:34.985790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:34.986741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:34.987590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] PATH_INFO: `/v3/projects/75f6261b90f54ee3b43dfa3364416528` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:34.990095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f6261b90f54ee3b43dfa3364416528)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:34.992421 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:35.073528 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f6261b90f54ee3b43dfa3364416528', 'request_id': 'req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.081295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f6261b90f54ee3b43dfa3364416528', 'request_id': 'req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.092448 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f6261b90f54ee3b43dfa3364416528', 'request_id': 'req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.119501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] The token cache is being invalidate because project 75f6261b90f54ee3b43dfa3364416528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:10:35.119825 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.120506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.127605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1053/2102] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:10:34 2022] DELETE /identity/v3/projects/75f6261b90f54ee3b43dfa3364416528 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:35.139405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:35.141270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:35.162653 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09108071-b97f-4d70-862b-b9c8819bc9f9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:35.242842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdbffe-f568-44d2-9d43-d6de3943ebd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:35.243809 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdbffe-f568-44d2-9d43-d6de3943ebd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:35.244207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdbffe-f568-44d2-9d43-d6de3943ebd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:35.244579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdbffe-f568-44d2-9d43-d6de3943ebd5 admin admin] PATH_INFO: `/v3/users/e6c2a3a1919145759718c8f7f1212aa0` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:35.245986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdbffe-f568-44d2-9d43-d6de3943ebd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6c2a3a1919145759718c8f7f1212aa0)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:35.248267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdbffe-f568-44d2-9d43-d6de3943ebd5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:35.316927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e9fdbffe-f568-44d2-9d43-d6de3943ebd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6c2a3a1919145759718c8f7f1212aa0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.317805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e9fdbffe-f568-44d2-9d43-d6de3943ebd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6c2a3a1919145759718c8f7f1212aa0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.327609 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e9fdbffe-f568-44d2-9d43-d6de3943ebd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6c2a3a1919145759718c8f7f1212aa0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.345732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e9fdbffe-f568-44d2-9d43-d6de3943ebd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6c2a3a1919145759718c8f7f1212aa0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.346438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e9fdbffe-f568-44d2-9d43-d6de3943ebd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6c2a3a1919145759718c8f7f1212aa0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.362642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1050/2103] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:10:35 2022] DELETE /identity/v3/users/e6c2a3a1919145759718c8f7f1212aa0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:35.666370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:35.668574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:35.686226 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a5ccf-a79c-4be0-85fa-a6ec448d9324 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:35.754708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acb736-ba04-4a58-9220-853024143dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:35.756050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acb736-ba04-4a58-9220-853024143dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:35.756699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acb736-ba04-4a58-9220-853024143dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:35.757216 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acb736-ba04-4a58-9220-853024143dd1 admin admin] PATH_INFO: `/v3/projects/c9ecc167515c4ce88616bfbef7461985` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:35.758696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58acb736-ba04-4a58-9220-853024143dd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9ecc167515c4ce88616bfbef7461985)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:35.761085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58acb736-ba04-4a58-9220-853024143dd1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:35.782003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:35.824868 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-58acb736-ba04-4a58-9220-853024143dd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9ecc167515c4ce88616bfbef7461985', 'request_id': 'req-58acb736-ba04-4a58-9220-853024143dd1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.831143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-58acb736-ba04-4a58-9220-853024143dd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9ecc167515c4ce88616bfbef7461985', 'request_id': 'req-58acb736-ba04-4a58-9220-853024143dd1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.838252 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-58acb736-ba04-4a58-9220-853024143dd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9ecc167515c4ce88616bfbef7461985', 'request_id': 'req-58acb736-ba04-4a58-9220-853024143dd1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.854329 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-58acb736-ba04-4a58-9220-853024143dd1 admin admin] The token cache is being invalidate because project c9ecc167515c4ce88616bfbef7461985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:10:35.854884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-58acb736-ba04-4a58-9220-853024143dd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.855510 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-58acb736-ba04-4a58-9220-853024143dd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:35.857307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1054/2104] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:10:35 2022] DELETE /identity/v3/projects/c9ecc167515c4ce88616bfbef7461985 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:35.863418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de7720-8b2c-4c88-8231-f669fb5692da None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:35.863976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de7720-8b2c-4c88-8231-f669fb5692da None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:35.864213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de7720-8b2c-4c88-8231-f669fb5692da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:35.878140 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:35.885989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-32de7720-8b2c-4c88-8231-f669fb5692da None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:35.888770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-32de7720-8b2c-4c88-8231-f669fb5692da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:35.963028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1051/2105] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:10:35.971638 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd770800-a1f6-4ebb-9f58-734ece4bd748 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:35.971927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd770800-a1f6-4ebb-9f58-734ece4bd748 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:35.972205 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd770800-a1f6-4ebb-9f58-734ece4bd748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:35.992058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-bd770800-a1f6-4ebb-9f58-734ece4bd748 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:35.993664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-bd770800-a1f6-4ebb-9f58-734ece4bd748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:36.004371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1055/2106] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:36.010026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de7720-8b2c-4c88-8231-f669fb5692da None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:36.020210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de7720-8b2c-4c88-8231-f669fb5692da None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:36.025550 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c4c9-5045-4f23-b8af-8c60384661eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:36.026652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c4c9-5045-4f23-b8af-8c60384661eb admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:36.026770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c4c9-5045-4f23-b8af-8c60384661eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:36.026987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c4c9-5045-4f23-b8af-8c60384661eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:36.027722 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7c4c9-5045-4f23-b8af-8c60384661eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:10:36.028081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7c4c9-5045-4f23-b8af-8c60384661eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:36.029914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7c4c9-5045-4f23-b8af-8c60384661eb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:36.044479 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1052/2107] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:10:36.049957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd770800-a1f6-4ebb-9f58-734ece4bd748 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:36.064682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd770800-a1f6-4ebb-9f58-734ece4bd748 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:36.071979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820a083-5c26-44a9-81f2-57ac6b8f68e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:36.072872 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820a083-5c26-44a9-81f2-57ac6b8f68e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:36.073247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820a083-5c26-44a9-81f2-57ac6b8f68e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:36.073644 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820a083-5c26-44a9-81f2-57ac6b8f68e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:36.074587 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820a083-5c26-44a9-81f2-57ac6b8f68e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:36.077308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820a083-5c26-44a9-81f2-57ac6b8f68e2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:36.117797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1056/2108] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:10:36 2022] POST /identity/v3/projects => generated 367 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:36.121854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c4c9-5045-4f23-b8af-8c60384661eb admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:36.140771 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c4c9-5045-4f23-b8af-8c60384661eb admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:36.218959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077190d7-2966-47dc-9eb7-6464df470dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:36.219975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077190d7-2966-47dc-9eb7-6464df470dfa admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:36.220360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077190d7-2966-47dc-9eb7-6464df470dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:36.220740 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077190d7-2966-47dc-9eb7-6464df470dfa admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:36.221856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077190d7-2966-47dc-9eb7-6464df470dfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:36.224344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077190d7-2966-47dc-9eb7-6464df470dfa admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:36.252888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1053/2109] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:10:36 2022] POST /identity/v3/users => generated 284 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:36.260455 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820a083-5c26-44a9-81f2-57ac6b8f68e2 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:36.277152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820a083-5c26-44a9-81f2-57ac6b8f68e2 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:36.287615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc072a18-e825-42bd-969b-3626177a3998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:36.288318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc072a18-e825-42bd-969b-3626177a3998 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:36.288604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc072a18-e825-42bd-969b-3626177a3998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:36.289506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc072a18-e825-42bd-969b-3626177a3998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:36.291403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc072a18-e825-42bd-969b-3626177a3998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:36.296883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc072a18-e825-42bd-969b-3626177a3998 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:36.332144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1057/2110] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:36.336882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077190d7-2966-47dc-9eb7-6464df470dfa admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:36.354791 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077190d7-2966-47dc-9eb7-6464df470dfa admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:36.361394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ec9d4-be6a-4c99-9ec9-fb20f4b5cde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:36.362738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ec9d4-be6a-4c99-9ec9-fb20f4b5cde2 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:36.363136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ec9d4-be6a-4c99-9ec9-fb20f4b5cde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:36.363519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ec9d4-be6a-4c99-9ec9-fb20f4b5cde2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:36.364913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ec9d4-be6a-4c99-9ec9-fb20f4b5cde2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:36.367628 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ec9d4-be6a-4c99-9ec9-fb20f4b5cde2 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:36.389235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1054/2111] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:36.390523 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc072a18-e825-42bd-969b-3626177a3998 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:36.401965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc072a18-e825-42bd-969b-3626177a3998 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:36.407495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718e6d4-6ea7-4f56-9a97-1899b5e960d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:36.408519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718e6d4-6ea7-4f56-9a97-1899b5e960d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:36.408855 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718e6d4-6ea7-4f56-9a97-1899b5e960d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:36.409200 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718e6d4-6ea7-4f56-9a97-1899b5e960d9 admin admin] PATH_INFO: `/v3/projects/e24c3f3ac6ab47789a40cad5950cf21e/users/c825c092e44844a28172c1909db7f58a/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:36.426094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718e6d4-6ea7-4f56-9a97-1899b5e960d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24c3f3ac6ab47789a40cad5950cf21e, user_id=c825c092e44844a28172c1909db7f58a, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:36.429182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718e6d4-6ea7-4f56-9a97-1899b5e960d9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:36.450192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1058/2112] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:36 2022] PUT /identity/v3/projects/e24c3f3ac6ab47789a40cad5950cf21e/users/c825c092e44844a28172c1909db7f58a/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:10:36.462969 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e92ff-1a53-4f01-83a3-64874d542e09 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:36.463271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e92ff-1a53-4f01-83a3-64874d542e09 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:36.463539 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e92ff-1a53-4f01-83a3-64874d542e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:36.496782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4a8e92ff-1a53-4f01-83a3-64874d542e09 None None] MFA Rules not processed for user `c825c092e44844a28172c1909db7f58a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:36.499094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4a8e92ff-1a53-4f01-83a3-64874d542e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:36.588810 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1055/2113] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:36.590223 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718e6d4-6ea7-4f56-9a97-1899b5e960d9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:36.699725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718e6d4-6ea7-4f56-9a97-1899b5e960d9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:36.704958 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e7e38-4700-4055-b0f6-dccb42f149cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:36.705783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e7e38-4700-4055-b0f6-dccb42f149cf admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:36.706163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e7e38-4700-4055-b0f6-dccb42f149cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:36.706536 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e7e38-4700-4055-b0f6-dccb42f149cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:36.707494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e7e38-4700-4055-b0f6-dccb42f149cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:36.709534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e7e38-4700-4055-b0f6-dccb42f149cf admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:36.742001 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1059/2114] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:10:36 2022] POST /identity/v3/projects => generated 367 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:36.746358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e92ff-1a53-4f01-83a3-64874d542e09 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:36.752805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:36.760280 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e92ff-1a53-4f01-83a3-64874d542e09 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:36.824110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010424a-3b3d-41b1-9b60-87f25a87a4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:36.825081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010424a-3b3d-41b1-9b60-87f25a87a4d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:36.825524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010424a-3b3d-41b1-9b60-87f25a87a4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:36.826069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010424a-3b3d-41b1-9b60-87f25a87a4d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:36.827119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010424a-3b3d-41b1-9b60-87f25a87a4d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:36.829324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010424a-3b3d-41b1-9b60-87f25a87a4d2 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:36.851714 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1056/2115] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:10:36 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:36.855760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e7e38-4700-4055-b0f6-dccb42f149cf admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:36.866414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e7e38-4700-4055-b0f6-dccb42f149cf admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:36.871270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f3f24-b209-4825-aa19-c4d5def9895e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:36.871929 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f3f24-b209-4825-aa19-c4d5def9895e admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:36.872206 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f3f24-b209-4825-aa19-c4d5def9895e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:36.872484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f3f24-b209-4825-aa19-c4d5def9895e admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:36.873228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f3f24-b209-4825-aa19-c4d5def9895e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:36.874874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f3f24-b209-4825-aa19-c4d5def9895e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:36.885816 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1060/2116] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:36.890097 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010424a-3b3d-41b1-9b60-87f25a87a4d2 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:36.903518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010424a-3b3d-41b1-9b60-87f25a87a4d2 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:36.908360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81449817-6b46-489c-9631-717e0e3e12b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:36.909304 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81449817-6b46-489c-9631-717e0e3e12b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:36.909661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81449817-6b46-489c-9631-717e0e3e12b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:36.910056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81449817-6b46-489c-9631-717e0e3e12b5 admin admin] PATH_INFO: `/v3/projects/83049f004b504642aa8cabfe484b129f/users/a5ce2361ef594bb8985adacbc41b6a3d/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:36.925818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81449817-6b46-489c-9631-717e0e3e12b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83049f004b504642aa8cabfe484b129f, user_id=a5ce2361ef594bb8985adacbc41b6a3d, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:36.928747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81449817-6b46-489c-9631-717e0e3e12b5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:36.944192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1057/2117] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:36 2022] PUT /identity/v3/projects/83049f004b504642aa8cabfe484b129f/users/a5ce2361ef594bb8985adacbc41b6a3d/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:36.948619 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f3f24-b209-4825-aa19-c4d5def9895e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:36.959760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f3f24-b209-4825-aa19-c4d5def9895e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:36.974614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:37.037082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f022a8-d8fc-4507-ad26-fbdb82244327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:37.037799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f022a8-d8fc-4507-ad26-fbdb82244327 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:37.038081 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f022a8-d8fc-4507-ad26-fbdb82244327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:37.038362 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f022a8-d8fc-4507-ad26-fbdb82244327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:37.039133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f022a8-d8fc-4507-ad26-fbdb82244327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:37.040815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f022a8-d8fc-4507-ad26-fbdb82244327 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:37.053678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1061/2118] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:37.056362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81449817-6b46-489c-9631-717e0e3e12b5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:37.068341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81449817-6b46-489c-9631-717e0e3e12b5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:37.073469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c698dab-ec82-4765-aeea-065d48fcafc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:37.074543 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c698dab-ec82-4765-aeea-065d48fcafc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:37.074944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c698dab-ec82-4765-aeea-065d48fcafc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:37.075311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c698dab-ec82-4765-aeea-065d48fcafc1 admin admin] PATH_INFO: `/v3/projects/83049f004b504642aa8cabfe484b129f/users/a5ce2361ef594bb8985adacbc41b6a3d/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:37.077637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c698dab-ec82-4765-aeea-065d48fcafc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83049f004b504642aa8cabfe484b129f, user_id=a5ce2361ef594bb8985adacbc41b6a3d, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:37.081359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c698dab-ec82-4765-aeea-065d48fcafc1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:37.095605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1058/2119] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:37 2022] PUT /identity/v3/projects/83049f004b504642aa8cabfe484b129f/users/a5ce2361ef594bb8985adacbc41b6a3d/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:10:37.105888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed71e8d-edc0-4d54-9815-4e3158684707 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:37.106209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed71e8d-edc0-4d54-9815-4e3158684707 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:37.106490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed71e8d-edc0-4d54-9815-4e3158684707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:37.151132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5ed71e8d-edc0-4d54-9815-4e3158684707 None None] MFA Rules not processed for user `a5ce2361ef594bb8985adacbc41b6a3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:37.154835 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5ed71e8d-edc0-4d54-9815-4e3158684707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:37.246284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1062/2120] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:37.265319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c698dab-ec82-4765-aeea-065d48fcafc1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:37.267152 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9c698dab-ec82-4765-aeea-065d48fcafc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:37.331519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c698dab-ec82-4765-aeea-065d48fcafc1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:37.336358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff99d4-bd81-4257-bc68-90f35740902f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:37.337321 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff99d4-bd81-4257-bc68-90f35740902f service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:37.337934 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff99d4-bd81-4257-bc68-90f35740902f service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:37.337934 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff99d4-bd81-4257-bc68-90f35740902f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:37.348619 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff99d4-bd81-4257-bc68-90f35740902f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:37.350236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff99d4-bd81-4257-bc68-90f35740902f service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:37.381751 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1059/2121] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:37.528934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed71e8d-edc0-4d54-9815-4e3158684707 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:37.555613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed71e8d-edc0-4d54-9815-4e3158684707 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:37.573316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb216828-550b-4598-ad55-32952b800f1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:37.574119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb216828-550b-4598-ad55-32952b800f1d service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:37.574403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb216828-550b-4598-ad55-32952b800f1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:37.574808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb216828-550b-4598-ad55-32952b800f1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:37.685726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb216828-550b-4598-ad55-32952b800f1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:37.688441 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb216828-550b-4598-ad55-32952b800f1d service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:37.705388 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1063/2122] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:43.145288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff99d4-bd81-4257-bc68-90f35740902f service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:43.146813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-03ff99d4-bd81-4257-bc68-90f35740902f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:43.259337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:43.261853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb216828-550b-4598-ad55-32952b800f1d service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:43.264130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff99d4-bd81-4257-bc68-90f35740902f service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:43.270964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3496b48-996b-4e7d-9106-a017c574b916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:43.272544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3496b48-996b-4e7d-9106-a017c574b916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:43.273081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3496b48-996b-4e7d-9106-a017c574b916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:43.280808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb216828-550b-4598-ad55-32952b800f1d service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:43.283431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3496b48-996b-4e7d-9106-a017c574b916 admin admin] PATH_INFO: `/v3/users/e38d9de21fe744bcb289f223d22076da` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:43.289797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3496b48-996b-4e7d-9106-a017c574b916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e38d9de21fe744bcb289f223d22076da)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:43.294246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6bb52-04a3-482b-bd12-53b20273bbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:43.295538 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6bb52-04a3-482b-bd12-53b20273bbea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:43.295992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6bb52-04a3-482b-bd12-53b20273bbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:43.296715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3496b48-996b-4e7d-9106-a017c574b916 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:43.297404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6bb52-04a3-482b-bd12-53b20273bbea admin admin] PATH_INFO: `/v3/users/c825c092e44844a28172c1909db7f58a` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:43.299126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b6bb52-04a3-482b-bd12-53b20273bbea admin admin] RBAC: Authorizing `identity:delete_user(user_id=c825c092e44844a28172c1909db7f58a)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:43.301420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b6bb52-04a3-482b-bd12-53b20273bbea admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:43.390093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:43.395606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d3496b48-996b-4e7d-9106-a017c574b916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e38d9de21fe744bcb289f223d22076da'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:43.396646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d3496b48-996b-4e7d-9106-a017c574b916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e38d9de21fe744bcb289f223d22076da'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:43.421439 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c7b6bb52-04a3-482b-bd12-53b20273bbea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c825c092e44844a28172c1909db7f58a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:43.422473 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c7b6bb52-04a3-482b-bd12-53b20273bbea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c825c092e44844a28172c1909db7f58a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:43.424656 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d3496b48-996b-4e7d-9106-a017c574b916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e38d9de21fe744bcb289f223d22076da'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:43.436771 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c7b6bb52-04a3-482b-bd12-53b20273bbea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c825c092e44844a28172c1909db7f58a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:43.459296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c7b6bb52-04a3-482b-bd12-53b20273bbea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c825c092e44844a28172c1909db7f58a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:43.461114 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c7b6bb52-04a3-482b-bd12-53b20273bbea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c825c092e44844a28172c1909db7f58a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:43.462123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d3496b48-996b-4e7d-9106-a017c574b916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e38d9de21fe744bcb289f223d22076da'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:43.463556 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d3496b48-996b-4e7d-9106-a017c574b916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e38d9de21fe744bcb289f223d22076da'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:43.482173 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1064/2123] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:10:43 2022] DELETE /identity/v3/users/c825c092e44844a28172c1909db7f58a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:43.487874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55487f4-a87e-44c2-8d4a-6eb34e26ea03 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:43.489204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55487f4-a87e-44c2-8d4a-6eb34e26ea03 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:43.491138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55487f4-a87e-44c2-8d4a-6eb34e26ea03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:43.501640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1060/2124] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:10:43 2022] DELETE /identity/v3/users/e38d9de21fe744bcb289f223d22076da => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:43.506215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef5a8-e9ae-4e5d-9954-ff0f117faff8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:43.506833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef5a8-e9ae-4e5d-9954-ff0f117faff8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:43.507162 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef5a8-e9ae-4e5d-9954-ff0f117faff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:43.523974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a55487f4-a87e-44c2-8d4a-6eb34e26ea03 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:43.525659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a55487f4-a87e-44c2-8d4a-6eb34e26ea03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:43.527686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9aaef5a8-e9ae-4e5d-9954-ff0f117faff8 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:43.528952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9aaef5a8-e9ae-4e5d-9954-ff0f117faff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:43.625227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1065/2125] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:43.635454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55487f4-a87e-44c2-8d4a-6eb34e26ea03 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:43.642467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a55487f4-a87e-44c2-8d4a-6eb34e26ea03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:43.644842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1061/2126] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:10:43.655004 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaef5a8-e9ae-4e5d-9954-ff0f117faff8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:43.656411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9aaef5a8-e9ae-4e5d-9954-ff0f117faff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:43.772795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaef5a8-e9ae-4e5d-9954-ff0f117faff8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:43.777010 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700d27f-b3f0-4405-ac6c-ed00cc57b0a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:43.778086 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700d27f-b3f0-4405-ac6c-ed00cc57b0a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:43.778605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700d27f-b3f0-4405-ac6c-ed00cc57b0a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:43.779744 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700d27f-b3f0-4405-ac6c-ed00cc57b0a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:43.781823 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55487f4-a87e-44c2-8d4a-6eb34e26ea03 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:43.792641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700d27f-b3f0-4405-ac6c-ed00cc57b0a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:43.792885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23da29-444a-468c-8506-d212bb4e4687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:43.793907 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23da29-444a-468c-8506-d212bb4e4687 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:43.794520 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23da29-444a-468c-8506-d212bb4e4687 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:43.794751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23da29-444a-468c-8506-d212bb4e4687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:43.795787 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700d27f-b3f0-4405-ac6c-ed00cc57b0a3 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:43.807292 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e23da29-444a-468c-8506-d212bb4e4687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:43.808760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e23da29-444a-468c-8506-d212bb4e4687 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:43.825673 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1066/2127] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:10:43.828762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1062/2128] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:10:44.279580 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23da29-444a-468c-8506-d212bb4e4687 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:44.282293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8e23da29-444a-468c-8506-d212bb4e4687 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:44.319406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23da29-444a-468c-8506-d212bb4e4687 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:44.330303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:44.331281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:44.331628 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:44.331985 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] PATH_INFO: `/v3/projects/e24c3f3ac6ab47789a40cad5950cf21e` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:44.334428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24c3f3ac6ab47789a40cad5950cf21e)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:44.336686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:44.404371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700d27f-b3f0-4405-ac6c-ed00cc57b0a3 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:44.405880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7700d27f-b3f0-4405-ac6c-ed00cc57b0a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:44.407417 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:44.410560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24c3f3ac6ab47789a40cad5950cf21e', 'request_id': 'req-8108ce92-51ce-4f9c-b693-65312df3407d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.420370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24c3f3ac6ab47789a40cad5950cf21e', 'request_id': 'req-8108ce92-51ce-4f9c-b693-65312df3407d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.430172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24c3f3ac6ab47789a40cad5950cf21e', 'request_id': 'req-8108ce92-51ce-4f9c-b693-65312df3407d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.440603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700d27f-b3f0-4405-ac6c-ed00cc57b0a3 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:44.450436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] The token cache is being invalidate because project e24c3f3ac6ab47789a40cad5950cf21e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:10:44.450687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.451335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.452984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1067/2129] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:10:44 2022] DELETE /identity/v3/projects/e24c3f3ac6ab47789a40cad5950cf21e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:44.458258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:44.459303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:44.467961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:44.534057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6307a-52a6-4208-9bbb-6fe530b3044a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:44.535428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6307a-52a6-4208-9bbb-6fe530b3044a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:44.535896 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6307a-52a6-4208-9bbb-6fe530b3044a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:44.536324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6307a-52a6-4208-9bbb-6fe530b3044a admin admin] PATH_INFO: `/v3/projects/f368d911b81a44dea97778601291db98` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:44.538991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6307a-52a6-4208-9bbb-6fe530b3044a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f368d911b81a44dea97778601291db98)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:44.540438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108ce92-51ce-4f9c-b693-65312df3407d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:44.541484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6307a-52a6-4208-9bbb-6fe530b3044a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:44.546596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:44.547497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:44.547783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:44.548059 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] PATH_INFO: `/v3/users/a5ce2361ef594bb8985adacbc41b6a3d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:44.549371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ce2361ef594bb8985adacbc41b6a3d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:44.552275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:44.621220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ce2361ef594bb8985adacbc41b6a3d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.622211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ce2361ef594bb8985adacbc41b6a3d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.625454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7cd6307a-52a6-4208-9bbb-6fe530b3044a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f368d911b81a44dea97778601291db98', 'request_id': 'req-7cd6307a-52a6-4208-9bbb-6fe530b3044a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.639930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ce2361ef594bb8985adacbc41b6a3d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.644527 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7cd6307a-52a6-4208-9bbb-6fe530b3044a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f368d911b81a44dea97778601291db98', 'request_id': 'req-7cd6307a-52a6-4208-9bbb-6fe530b3044a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.653220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7cd6307a-52a6-4208-9bbb-6fe530b3044a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f368d911b81a44dea97778601291db98', 'request_id': 'req-7cd6307a-52a6-4208-9bbb-6fe530b3044a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.658711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ce2361ef594bb8985adacbc41b6a3d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.659751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ce2361ef594bb8985adacbc41b6a3d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.679126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1068/2130] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:10:44 2022] DELETE /identity/v3/users/a5ce2361ef594bb8985adacbc41b6a3d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:44.687085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7cd6307a-52a6-4208-9bbb-6fe530b3044a admin admin] The token cache is being invalidate because project f368d911b81a44dea97778601291db98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:10:44.687455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7cd6307a-52a6-4208-9bbb-6fe530b3044a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.688290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7cd6307a-52a6-4208-9bbb-6fe530b3044a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:44.690267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1063/2131] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:10:44 2022] DELETE /identity/v3/projects/f368d911b81a44dea97778601291db98 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:10:45.250246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:45.251477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:45.346892 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:45.365997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aef510-4510-4081-a991-1f0ae84805dc admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:45.373438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb2417-78d5-4ef9-95dc-763038264267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:45.374577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb2417-78d5-4ef9-95dc-763038264267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:45.374703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb2417-78d5-4ef9-95dc-763038264267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:45.374965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb2417-78d5-4ef9-95dc-763038264267 admin admin] PATH_INFO: `/v3/projects/83049f004b504642aa8cabfe484b129f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:45.376588 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb2417-78d5-4ef9-95dc-763038264267 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83049f004b504642aa8cabfe484b129f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:45.378703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb2417-78d5-4ef9-95dc-763038264267 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:45.484933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b4bb2417-78d5-4ef9-95dc-763038264267 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83049f004b504642aa8cabfe484b129f', 'request_id': 'req-b4bb2417-78d5-4ef9-95dc-763038264267'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:45.496971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b4bb2417-78d5-4ef9-95dc-763038264267 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83049f004b504642aa8cabfe484b129f', 'request_id': 'req-b4bb2417-78d5-4ef9-95dc-763038264267'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:45.505446 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b4bb2417-78d5-4ef9-95dc-763038264267 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83049f004b504642aa8cabfe484b129f', 'request_id': 'req-b4bb2417-78d5-4ef9-95dc-763038264267'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:45.521987 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b4bb2417-78d5-4ef9-95dc-763038264267 admin admin] The token cache is being invalidate because project 83049f004b504642aa8cabfe484b129f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:10:45.522268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b4bb2417-78d5-4ef9-95dc-763038264267 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:45.522940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b4bb2417-78d5-4ef9-95dc-763038264267 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:45.524643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1069/2132] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:10:45 2022] DELETE /identity/v3/projects/83049f004b504642aa8cabfe484b129f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:45.529951 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee993d7-fc6e-4cb7-8b9f-275e957e8e30 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:45.530498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee993d7-fc6e-4cb7-8b9f-275e957e8e30 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:45.530498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee993d7-fc6e-4cb7-8b9f-275e957e8e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:45.551147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-aee993d7-fc6e-4cb7-8b9f-275e957e8e30 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:45.552434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-aee993d7-fc6e-4cb7-8b9f-275e957e8e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:45.623353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1064/2133] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:45.632007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa7c43-b52c-4ac1-8fce-4108b4cd84fb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:45.632316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa7c43-b52c-4ac1-8fce-4108b4cd84fb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:45.632575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa7c43-b52c-4ac1-8fce-4108b4cd84fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:45.663265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f6aa7c43-b52c-4ac1-8fce-4108b4cd84fb None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:45.664540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f6aa7c43-b52c-4ac1-8fce-4108b4cd84fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:45.678846 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1070/2134] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:45.680536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee993d7-fc6e-4cb7-8b9f-275e957e8e30 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:45.692185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee993d7-fc6e-4cb7-8b9f-275e957e8e30 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:45.697251 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323eb2a-5b2f-4146-a523-b0f6db6604cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:45.697983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323eb2a-5b2f-4146-a523-b0f6db6604cb admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:45.698270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323eb2a-5b2f-4146-a523-b0f6db6604cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:45.698552 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323eb2a-5b2f-4146-a523-b0f6db6604cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:45.699330 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323eb2a-5b2f-4146-a523-b0f6db6604cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:10:45.699650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323eb2a-5b2f-4146-a523-b0f6db6604cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:45.701339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323eb2a-5b2f-4146-a523-b0f6db6604cb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:45.717203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1065/2135] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:10:45.718738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa7c43-b52c-4ac1-8fce-4108b4cd84fb None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:45.746950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa7c43-b52c-4ac1-8fce-4108b4cd84fb None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:45.746950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4238e-8b0b-4abd-a90d-076f989d8a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:45.747411 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4238e-8b0b-4abd-a90d-076f989d8a18 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:45.747411 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4238e-8b0b-4abd-a90d-076f989d8a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:45.747691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4238e-8b0b-4abd-a90d-076f989d8a18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:45.748995 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4238e-8b0b-4abd-a90d-076f989d8a18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:45.751248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4238e-8b0b-4abd-a90d-076f989d8a18 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:45.786992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1071/2136] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:10:45 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:10:45.788362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323eb2a-5b2f-4146-a523-b0f6db6604cb admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:45.798903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323eb2a-5b2f-4146-a523-b0f6db6604cb admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:45.840421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:45.884584 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f814cc0-deac-4cfc-8bda-e4e4f696266e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:45.886490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f814cc0-deac-4cfc-8bda-e4e4f696266e admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:45.889730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f814cc0-deac-4cfc-8bda-e4e4f696266e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:45.890217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f814cc0-deac-4cfc-8bda-e4e4f696266e admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:45.891671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f814cc0-deac-4cfc-8bda-e4e4f696266e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:45.896582 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f814cc0-deac-4cfc-8bda-e4e4f696266e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:45.929724 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1066/2137] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:10:45 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:45.930961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4238e-8b0b-4abd-a90d-076f989d8a18 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:45.958706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4238e-8b0b-4abd-a90d-076f989d8a18 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:45.966209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854152fc-edd4-4a69-9919-d4b659f1d7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:45.967543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854152fc-edd4-4a69-9919-d4b659f1d7cc admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:45.968042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854152fc-edd4-4a69-9919-d4b659f1d7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:45.969229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854152fc-edd4-4a69-9919-d4b659f1d7cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:45.971001 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854152fc-edd4-4a69-9919-d4b659f1d7cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:45.973399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854152fc-edd4-4a69-9919-d4b659f1d7cc admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:45.999778 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1072/2138] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:45.999872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f814cc0-deac-4cfc-8bda-e4e4f696266e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:46.006398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f814cc0-deac-4cfc-8bda-e4e4f696266e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:46.017267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb810b-3895-494a-bb04-85fe1aec3902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:46.018054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb810b-3895-494a-bb04-85fe1aec3902 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:46.018514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb810b-3895-494a-bb04-85fe1aec3902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:46.018799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb810b-3895-494a-bb04-85fe1aec3902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:46.019550 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb810b-3895-494a-bb04-85fe1aec3902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:46.021197 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb810b-3895-494a-bb04-85fe1aec3902 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:46.035641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1067/2139] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:46.036949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854152fc-edd4-4a69-9919-d4b659f1d7cc admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:46.050656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854152fc-edd4-4a69-9919-d4b659f1d7cc admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:46.060926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fbde0-8b68-425a-9c37-9c0658ea7fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:46.062468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fbde0-8b68-425a-9c37-9c0658ea7fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:46.062830 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fbde0-8b68-425a-9c37-9c0658ea7fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:46.063180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fbde0-8b68-425a-9c37-9c0658ea7fbd admin admin] PATH_INFO: `/v3/projects/c8fb3aa91110442c97bceb3f0845a6d7/users/b78ba32a1e5a4497a5a13c8e5f591f67/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:46.094523 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fbde0-8b68-425a-9c37-9c0658ea7fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8fb3aa91110442c97bceb3f0845a6d7, user_id=b78ba32a1e5a4497a5a13c8e5f591f67, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:46.098805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fbde0-8b68-425a-9c37-9c0658ea7fbd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:46.149852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1073/2140] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:46 2022] PUT /identity/v3/projects/c8fb3aa91110442c97bceb3f0845a6d7/users/b78ba32a1e5a4497a5a13c8e5f591f67/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:46.161998 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708fd66-b577-4568-803f-e8971d00b93e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:46.162250 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708fd66-b577-4568-803f-e8971d00b93e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:46.162681 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708fd66-b577-4568-803f-e8971d00b93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:46.209139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5708fd66-b577-4568-803f-e8971d00b93e None None] MFA Rules not processed for user `b78ba32a1e5a4497a5a13c8e5f591f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:46.210505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5708fd66-b577-4568-803f-e8971d00b93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:46.334756 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1068/2141] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:46.340484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fbde0-8b68-425a-9c37-9c0658ea7fbd admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:46.425608 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fbde0-8b68-425a-9c37-9c0658ea7fbd admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:46.431453 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc88c90-9814-4c43-b20f-c902aeb2e767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:46.432271 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc88c90-9814-4c43-b20f-c902aeb2e767 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:46.432937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc88c90-9814-4c43-b20f-c902aeb2e767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:46.432937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc88c90-9814-4c43-b20f-c902aeb2e767 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:46.433928 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc88c90-9814-4c43-b20f-c902aeb2e767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:46.436042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc88c90-9814-4c43-b20f-c902aeb2e767 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:46.481394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1074/2142] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:10:46 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:46.482617 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708fd66-b577-4568-803f-e8971d00b93e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:46.499165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708fd66-b577-4568-803f-e8971d00b93e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:46.578345 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3feac-aa7c-4a1a-92c3-12b5653f0580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:46.579096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3feac-aa7c-4a1a-92c3-12b5653f0580 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:46.579298 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3feac-aa7c-4a1a-92c3-12b5653f0580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:46.579584 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3feac-aa7c-4a1a-92c3-12b5653f0580 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:46.580290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3feac-aa7c-4a1a-92c3-12b5653f0580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:46.582286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3feac-aa7c-4a1a-92c3-12b5653f0580 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:46.605931 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1069/2143] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:10:46 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:46.610128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc88c90-9814-4c43-b20f-c902aeb2e767 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:46.617759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:46.622003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc88c90-9814-4c43-b20f-c902aeb2e767 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:46.632346 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207cbb7-398f-4003-b70a-db3fab57502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:46.633357 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207cbb7-398f-4003-b70a-db3fab57502a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:46.633892 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207cbb7-398f-4003-b70a-db3fab57502a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:46.634412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207cbb7-398f-4003-b70a-db3fab57502a admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:46.635202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207cbb7-398f-4003-b70a-db3fab57502a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:46.636879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207cbb7-398f-4003-b70a-db3fab57502a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:46.649062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1075/2144] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:46.653238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3feac-aa7c-4a1a-92c3-12b5653f0580 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:46.664293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3feac-aa7c-4a1a-92c3-12b5653f0580 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:46.668944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8bace-f68d-4fc0-9c32-89fe6505dd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:46.670117 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8bace-f68d-4fc0-9c32-89fe6505dd9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:46.670399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8bace-f68d-4fc0-9c32-89fe6505dd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:46.670675 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8bace-f68d-4fc0-9c32-89fe6505dd9a admin admin] PATH_INFO: `/v3/projects/d4147ff3ed4b45da8df1f30820b349ae/users/a2d34997e11a4c8384ad104a1dab60c8/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:46.686885 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8bace-f68d-4fc0-9c32-89fe6505dd9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4147ff3ed4b45da8df1f30820b349ae, user_id=a2d34997e11a4c8384ad104a1dab60c8, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:46.689745 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8bace-f68d-4fc0-9c32-89fe6505dd9a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:46.706149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1070/2145] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:46 2022] PUT /identity/v3/projects/d4147ff3ed4b45da8df1f30820b349ae/users/a2d34997e11a4c8384ad104a1dab60c8/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:46.714184 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207cbb7-398f-4003-b70a-db3fab57502a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:46.725041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207cbb7-398f-4003-b70a-db3fab57502a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:46.798372 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626050a-072e-405c-974a-d384d1cb1ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:46.799097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626050a-072e-405c-974a-d384d1cb1ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:46.799384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626050a-072e-405c-974a-d384d1cb1ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:46.799660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626050a-072e-405c-974a-d384d1cb1ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:46.800394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626050a-072e-405c-974a-d384d1cb1ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:46.802124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626050a-072e-405c-974a-d384d1cb1ba0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:46.818472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1076/2146] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:46.819700 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8bace-f68d-4fc0-9c32-89fe6505dd9a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:46.830753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8bace-f68d-4fc0-9c32-89fe6505dd9a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:46.835517 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f8d6f-196b-41e1-a964-9b49e5341462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:46.836458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8d6f-196b-41e1-a964-9b49e5341462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:46.836777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8d6f-196b-41e1-a964-9b49e5341462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:46.837060 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8d6f-196b-41e1-a964-9b49e5341462 admin admin] PATH_INFO: `/v3/projects/d4147ff3ed4b45da8df1f30820b349ae/users/a2d34997e11a4c8384ad104a1dab60c8/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:46.839519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f8d6f-196b-41e1-a964-9b49e5341462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4147ff3ed4b45da8df1f30820b349ae, user_id=a2d34997e11a4c8384ad104a1dab60c8, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:46.843043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f8d6f-196b-41e1-a964-9b49e5341462 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:46.850319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:46.857284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1071/2147] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:46 2022] PUT /identity/v3/projects/d4147ff3ed4b45da8df1f30820b349ae/users/a2d34997e11a4c8384ad104a1dab60c8/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:46.867725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe3798-70c8-4ea0-93b7-8228d65d3e35 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:46.868022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe3798-70c8-4ea0-93b7-8228d65d3e35 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:46.868298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe3798-70c8-4ea0-93b7-8228d65d3e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:46.901190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-dbbe3798-70c8-4ea0-93b7-8228d65d3e35 None None] MFA Rules not processed for user `a2d34997e11a4c8384ad104a1dab60c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:46.902358 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-dbbe3798-70c8-4ea0-93b7-8228d65d3e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:46.980971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1077/2148] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:46.992237 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f8d6f-196b-41e1-a964-9b49e5341462 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:46.994072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-be9f8d6f-196b-41e1-a964-9b49e5341462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:47.059456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f8d6f-196b-41e1-a964-9b49e5341462 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:47.064331 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdf626-d24f-4222-a750-f22de0821dd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:47.064984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf626-d24f-4222-a750-f22de0821dd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:47.065264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf626-d24f-4222-a750-f22de0821dd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:47.065538 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf626-d24f-4222-a750-f22de0821dd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:47.075402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdf626-d24f-4222-a750-f22de0821dd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:47.076956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdf626-d24f-4222-a750-f22de0821dd0 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:47.086467 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1072/2149] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:47.700870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe3798-70c8-4ea0-93b7-8228d65d3e35 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:47.773304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe3798-70c8-4ea0-93b7-8228d65d3e35 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:47.778005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:47.779117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:47.779394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:47.779665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] PATH_INFO: `/v3/users/b78ba32a1e5a4497a5a13c8e5f591f67` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:47.780875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b78ba32a1e5a4497a5a13c8e5f591f67)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:47.782870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:47.798454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:47.832761 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b78ba32a1e5a4497a5a13c8e5f591f67'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:47.833440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b78ba32a1e5a4497a5a13c8e5f591f67'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:47.842289 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b78ba32a1e5a4497a5a13c8e5f591f67'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:47.855685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b78ba32a1e5a4497a5a13c8e5f591f67'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:47.856366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b78ba32a1e5a4497a5a13c8e5f591f67'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:47.874457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1078/2150] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:10:47 2022] DELETE /identity/v3/users/b78ba32a1e5a4497a5a13c8e5f591f67 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:47.880338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c0925-f5d3-47c2-8530-75ea14a7637e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:47.880819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c0925-f5d3-47c2-8530-75ea14a7637e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:47.880905 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c0925-f5d3-47c2-8530-75ea14a7637e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:47.899538 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5b7c0925-f5d3-47c2-8530-75ea14a7637e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:47.900720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5b7c0925-f5d3-47c2-8530-75ea14a7637e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:47.961774 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1073/2151] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:47.974803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:47.976471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:48.043248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e948f-5038-43b8-b18c-ae2705ff51d5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:48.047793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85cac5-6b39-489b-8a42-40f34a381eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:48.048467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85cac5-6b39-489b-8a42-40f34a381eba service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:48.048751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85cac5-6b39-489b-8a42-40f34a381eba service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:48.049027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85cac5-6b39-489b-8a42-40f34a381eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:48.059689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85cac5-6b39-489b-8a42-40f34a381eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:48.061190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85cac5-6b39-489b-8a42-40f34a381eba service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:48.072248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1079/2152] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:48.206065 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad31f8e-11b5-4f3b-aa33-bdfaa4067a86 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:48.206065 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad31f8e-11b5-4f3b-aa33-bdfaa4067a86 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:48.206286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad31f8e-11b5-4f3b-aa33-bdfaa4067a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:48.224931 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2ad31f8e-11b5-4f3b-aa33-bdfaa4067a86 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:48.226772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2ad31f8e-11b5-4f3b-aa33-bdfaa4067a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:48.302245 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1074/2153] 10.222.0.79 () {58 vars in 967 bytes} [Mon Jul 4 10:10:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:48.315223 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85cac5-6b39-489b-8a42-40f34a381eba service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:48.316678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4d85cac5-6b39-489b-8a42-40f34a381eba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:48.370813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad31f8e-11b5-4f3b-aa33-bdfaa4067a86 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:48.372539 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2ad31f8e-11b5-4f3b-aa33-bdfaa4067a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:48.387743 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad31f8e-11b5-4f3b-aa33-bdfaa4067a86 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:48.393101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:48.394028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:48.394316 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:48.394589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] PATH_INFO: `/v3/projects/c8fb3aa91110442c97bceb3f0845a6d7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:48.395830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8fb3aa91110442c97bceb3f0845a6d7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:48.397536 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85cac5-6b39-489b-8a42-40f34a381eba service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:48.397804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:48.404530 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed6b85-ba6d-4b51-8e41-dfc6cedc8805 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:48.404997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed6b85-ba6d-4b51-8e41-dfc6cedc8805 service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:48.405355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed6b85-ba6d-4b51-8e41-dfc6cedc8805 service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:48.405819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed6b85-ba6d-4b51-8e41-dfc6cedc8805 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:48.415942 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed6b85-ba6d-4b51-8e41-dfc6cedc8805 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:48.417575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed6b85-ba6d-4b51-8e41-dfc6cedc8805 service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:48.426623 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1080/2154] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:10:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 10:10:48.442457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:48.449341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8fb3aa91110442c97bceb3f0845a6d7', 'request_id': 'req-315d7a6f-5eea-45bd-8232-3efb05d5d527'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:48.456958 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8fb3aa91110442c97bceb3f0845a6d7', 'request_id': 'req-315d7a6f-5eea-45bd-8232-3efb05d5d527'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:48.464824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8fb3aa91110442c97bceb3f0845a6d7', 'request_id': 'req-315d7a6f-5eea-45bd-8232-3efb05d5d527'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:48.483851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] The token cache is being invalidate because project c8fb3aa91110442c97bceb3f0845a6d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:10:48.484332 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:48.485078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:48.487279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1075/2155] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:10:48 2022] DELETE /identity/v3/projects/c8fb3aa91110442c97bceb3f0845a6d7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:10:48.490993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed6b85-ba6d-4b51-8e41-dfc6cedc8805 service swift] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:48.503280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed6b85-ba6d-4b51-8e41-dfc6cedc8805 service swift] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:48.532967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:48.569637 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012320d2-91d5-43cb-bcfd-670c0ffe2c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:48.570615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012320d2-91d5-43cb-bcfd-670c0ffe2c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:48.570900 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012320d2-91d5-43cb-bcfd-670c0ffe2c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:48.571181 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012320d2-91d5-43cb-bcfd-670c0ffe2c3c admin admin] PATH_INFO: `/v3/users/a2d34997e11a4c8384ad104a1dab60c8` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:48.572414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012320d2-91d5-43cb-bcfd-670c0ffe2c3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2d34997e11a4c8384ad104a1dab60c8)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:48.574439 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012320d2-91d5-43cb-bcfd-670c0ffe2c3c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:48.644815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-012320d2-91d5-43cb-bcfd-670c0ffe2c3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2d34997e11a4c8384ad104a1dab60c8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:48.645489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-012320d2-91d5-43cb-bcfd-670c0ffe2c3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2d34997e11a4c8384ad104a1dab60c8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:48.654257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-012320d2-91d5-43cb-bcfd-670c0ffe2c3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2d34997e11a4c8384ad104a1dab60c8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:48.670471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-012320d2-91d5-43cb-bcfd-670c0ffe2c3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2d34997e11a4c8384ad104a1dab60c8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:48.671257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-012320d2-91d5-43cb-bcfd-670c0ffe2c3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2d34997e11a4c8384ad104a1dab60c8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:48.686843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1081/2156] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:10:48 2022] DELETE /identity/v3/users/a2d34997e11a4c8384ad104a1dab60c8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:49.108862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:49.110432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:49.135363 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d7a6f-5eea-45bd-8232-3efb05d5d527 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:49.194089 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:49.194971 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:49.195255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:49.195535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81 admin admin] PATH_INFO: `/v3/projects/d4147ff3ed4b45da8df1f30820b349ae` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:49.196728 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4147ff3ed4b45da8df1f30820b349ae)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:49.198740 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:49.248894 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4147ff3ed4b45da8df1f30820b349ae', 'request_id': 'req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:49.255082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4147ff3ed4b45da8df1f30820b349ae', 'request_id': 'req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:49.260678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4147ff3ed4b45da8df1f30820b349ae', 'request_id': 'req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:49.275130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81 admin admin] The token cache is being invalidate because project d4147ff3ed4b45da8df1f30820b349ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:10:49.276029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:49.276713 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6d439a15-fdfa-4025-a1d7-cd1ce610fe81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:49.278279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1076/2157] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:10:49 2022] DELETE /identity/v3/projects/d4147ff3ed4b45da8df1f30820b349ae => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:10:49.283428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bc24a-e971-4f4e-b317-d30d2f98aff5 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:49.283944 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bc24a-e971-4f4e-b317-d30d2f98aff5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:49.284048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bc24a-e971-4f4e-b317-d30d2f98aff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:49.302087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1c4bc24a-e971-4f4e-b317-d30d2f98aff5 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:49.303297 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1c4bc24a-e971-4f4e-b317-d30d2f98aff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:49.364817 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1082/2158] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:49.373895 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65875adc-7217-4fab-a72a-a7bd8f585bbc None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:49.374144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65875adc-7217-4fab-a72a-a7bd8f585bbc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:49.374456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65875adc-7217-4fab-a72a-a7bd8f585bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:49.393303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-65875adc-7217-4fab-a72a-a7bd8f585bbc None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:49.394561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-65875adc-7217-4fab-a72a-a7bd8f585bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:49.403630 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1077/2159] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:10:49.407949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bc24a-e971-4f4e-b317-d30d2f98aff5 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:49.417786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bc24a-e971-4f4e-b317-d30d2f98aff5 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:49.422900 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca648e36-ae33-4c81-bbd6-1a69e1317955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:49.423572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca648e36-ae33-4c81-bbd6-1a69e1317955 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:49.423854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca648e36-ae33-4c81-bbd6-1a69e1317955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:49.424129 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca648e36-ae33-4c81-bbd6-1a69e1317955 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:49.424864 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca648e36-ae33-4c81-bbd6-1a69e1317955 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:10:49.425170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca648e36-ae33-4c81-bbd6-1a69e1317955 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:49.426822 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca648e36-ae33-4c81-bbd6-1a69e1317955 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:49.438586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1083/2160] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:10:49.443041 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65875adc-7217-4fab-a72a-a7bd8f585bbc None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:49.452586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65875adc-7217-4fab-a72a-a7bd8f585bbc None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:49.458339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb288df-9b02-4357-bf80-3b5eab495ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:49.458985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb288df-9b02-4357-bf80-3b5eab495ffe admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:49.459265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb288df-9b02-4357-bf80-3b5eab495ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:49.459538 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb288df-9b02-4357-bf80-3b5eab495ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:49.460226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb288df-9b02-4357-bf80-3b5eab495ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:49.462181 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb288df-9b02-4357-bf80-3b5eab495ffe admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:49.492415 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1078/2161] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:10:49 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:49.497348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca648e36-ae33-4c81-bbd6-1a69e1317955 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:49.507514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca648e36-ae33-4c81-bbd6-1a69e1317955 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:49.538971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:49.568594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6568770-7662-4d3e-b0ef-c0c94dcc70da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:49.569267 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6568770-7662-4d3e-b0ef-c0c94dcc70da admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:49.569529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6568770-7662-4d3e-b0ef-c0c94dcc70da admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:49.569842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6568770-7662-4d3e-b0ef-c0c94dcc70da admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:49.570540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6568770-7662-4d3e-b0ef-c0c94dcc70da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:49.572612 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6568770-7662-4d3e-b0ef-c0c94dcc70da admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:49.595790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1084/2162] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:10:49 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:49.600270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb288df-9b02-4357-bf80-3b5eab495ffe admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:49.611175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb288df-9b02-4357-bf80-3b5eab495ffe admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:49.615940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa67f66-2939-4b85-8069-abd38a2f96f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:49.616603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa67f66-2939-4b85-8069-abd38a2f96f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:49.616895 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa67f66-2939-4b85-8069-abd38a2f96f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:49.617172 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa67f66-2939-4b85-8069-abd38a2f96f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:49.617945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa67f66-2939-4b85-8069-abd38a2f96f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:49.619592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa67f66-2939-4b85-8069-abd38a2f96f5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:49.629888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1079/2163] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:10:49.633910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6568770-7662-4d3e-b0ef-c0c94dcc70da admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:49.644148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6568770-7662-4d3e-b0ef-c0c94dcc70da admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:49.649035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075327e-0a06-472c-8ce0-f841f5ce77b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:49.649717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075327e-0a06-472c-8ce0-f841f5ce77b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:49.650001 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075327e-0a06-472c-8ce0-f841f5ce77b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:49.650275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075327e-0a06-472c-8ce0-f841f5ce77b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:49.651009 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075327e-0a06-472c-8ce0-f841f5ce77b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:49.652617 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075327e-0a06-472c-8ce0-f841f5ce77b9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:49.664219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1085/2164] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:49.668136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa67f66-2939-4b85-8069-abd38a2f96f5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:49.678414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa67f66-2939-4b85-8069-abd38a2f96f5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:49.683301 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6685b-173f-458f-93cb-64c50442600d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:49.684186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6685b-173f-458f-93cb-64c50442600d admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:49.684469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6685b-173f-458f-93cb-64c50442600d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:49.684797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6685b-173f-458f-93cb-64c50442600d admin admin] PATH_INFO: `/v3/projects/48aaf84d03a347bcb6ac42ca49b138cf/users/f1f5120a913c4a6db1a9eef0eebbeaa0/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:49.701149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6685b-173f-458f-93cb-64c50442600d admin admin] RBAC: Authorizing `identity:create_grant(project_id=48aaf84d03a347bcb6ac42ca49b138cf, user_id=f1f5120a913c4a6db1a9eef0eebbeaa0, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:49.703871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6685b-173f-458f-93cb-64c50442600d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:49.707179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:49.724199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1080/2165] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:49 2022] PUT /identity/v3/projects/48aaf84d03a347bcb6ac42ca49b138cf/users/f1f5120a913c4a6db1a9eef0eebbeaa0/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:49.729270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfae41-2cea-49cc-9b74-faacddfe04a2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:49.729646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfae41-2cea-49cc-9b74-faacddfe04a2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:49.729993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfae41-2cea-49cc-9b74-faacddfe04a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:49.748154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-42cfae41-2cea-49cc-9b74-faacddfe04a2 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:49.749418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-42cfae41-2cea-49cc-9b74-faacddfe04a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:49.812055 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1086/2166] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:10:49.821947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6685b-173f-458f-93cb-64c50442600d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:49.823915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-06c6685b-173f-458f-93cb-64c50442600d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:49.894523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6685b-173f-458f-93cb-64c50442600d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:49.901285 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4debddb4-7a85-4c52-8624-5ba2b4ab1051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:49.902220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debddb4-7a85-4c52-8624-5ba2b4ab1051 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:49.902596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debddb4-7a85-4c52-8624-5ba2b4ab1051 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:49.902910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debddb4-7a85-4c52-8624-5ba2b4ab1051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:49.912945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4debddb4-7a85-4c52-8624-5ba2b4ab1051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:49.914408 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4debddb4-7a85-4c52-8624-5ba2b4ab1051 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:49.923867 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1081/2167] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:50.585084 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfae41-2cea-49cc-9b74-faacddfe04a2 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:50.587047 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-42cfae41-2cea-49cc-9b74-faacddfe04a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:50.602687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfae41-2cea-49cc-9b74-faacddfe04a2 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:50.610090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:50.611029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:50.611306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:50.611627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] PATH_INFO: `/v3/users/348d6a68d7c84200899be9e72aeaded6` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:50.612799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=348d6a68d7c84200899be9e72aeaded6)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:50.614799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:50.669265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348d6a68d7c84200899be9e72aeaded6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:50.670121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348d6a68d7c84200899be9e72aeaded6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:50.679101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348d6a68d7c84200899be9e72aeaded6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:50.692084 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348d6a68d7c84200899be9e72aeaded6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:50.692769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348d6a68d7c84200899be9e72aeaded6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:50.708406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1087/2168] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:10:50 2022] DELETE /identity/v3/users/348d6a68d7c84200899be9e72aeaded6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:50.842779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cec30-74fa-4a96-a4bf-c9076f89b33f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:50.843552 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cec30-74fa-4a96-a4bf-c9076f89b33f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:50.844743 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cec30-74fa-4a96-a4bf-c9076f89b33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:50.890484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-547cec30-74fa-4a96-a4bf-c9076f89b33f None None] MFA Rules not processed for user `f1f5120a913c4a6db1a9eef0eebbeaa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:50.891849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-547cec30-74fa-4a96-a4bf-c9076f89b33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:50.902835 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:50.904811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:50.953274 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:50.996727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1082/2169] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:51.002751 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cec30-74fa-4a96-a4bf-c9076f89b33f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:51.006216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-547cec30-74fa-4a96-a4bf-c9076f89b33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:51.008959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5df1f-79f9-4a86-afa3-071df174f6b9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.013926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251eb0b1-ecda-47bd-869a-fb21c2c256a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.014869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eb0b1-ecda-47bd-869a-fb21c2c256a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.015220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eb0b1-ecda-47bd-869a-fb21c2c256a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.016295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eb0b1-ecda-47bd-869a-fb21c2c256a1 admin admin] PATH_INFO: `/v3/projects/59fcbc120f0d4dc48dc0b3f2e7433b1a` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.017236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251eb0b1-ecda-47bd-869a-fb21c2c256a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59fcbc120f0d4dc48dc0b3f2e7433b1a)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.019283 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251eb0b1-ecda-47bd-869a-fb21c2c256a1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.028845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cec30-74fa-4a96-a4bf-c9076f89b33f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.035173 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb56baf-a4f5-4acc-b91f-37fcd5104654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.036038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb56baf-a4f5-4acc-b91f-37fcd5104654 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.036328 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb56baf-a4f5-4acc-b91f-37fcd5104654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.036608 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb56baf-a4f5-4acc-b91f-37fcd5104654 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.038811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb56baf-a4f5-4acc-b91f-37fcd5104654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.040765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb56baf-a4f5-4acc-b91f-37fcd5104654 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.078296 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1083/2170] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:10:51 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:51.082219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb56baf-a4f5-4acc-b91f-37fcd5104654 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:51.086102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-251eb0b1-ecda-47bd-869a-fb21c2c256a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59fcbc120f0d4dc48dc0b3f2e7433b1a', 'request_id': 'req-251eb0b1-ecda-47bd-869a-fb21c2c256a1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:51.091977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-251eb0b1-ecda-47bd-869a-fb21c2c256a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59fcbc120f0d4dc48dc0b3f2e7433b1a', 'request_id': 'req-251eb0b1-ecda-47bd-869a-fb21c2c256a1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:51.098992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-251eb0b1-ecda-47bd-869a-fb21c2c256a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59fcbc120f0d4dc48dc0b3f2e7433b1a', 'request_id': 'req-251eb0b1-ecda-47bd-869a-fb21c2c256a1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:51.114354 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-251eb0b1-ecda-47bd-869a-fb21c2c256a1 admin admin] The token cache is being invalidate because project 59fcbc120f0d4dc48dc0b3f2e7433b1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:10:51.114642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-251eb0b1-ecda-47bd-869a-fb21c2c256a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:51.115274 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-251eb0b1-ecda-47bd-869a-fb21c2c256a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:10:51.116905 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1088/2171] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:10:50 2022] DELETE /identity/v3/projects/59fcbc120f0d4dc48dc0b3f2e7433b1a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:10:51.125304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ec993-d53f-4307-868f-d64060dabdfa None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.125304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ec993-d53f-4307-868f-d64060dabdfa None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.125304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ec993-d53f-4307-868f-d64060dabdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.145883 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb56baf-a4f5-4acc-b91f-37fcd5104654 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.146781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f57ec993-d53f-4307-868f-d64060dabdfa None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:51.148071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f57ec993-d53f-4307-868f-d64060dabdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:51.151369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f54c3-c75d-4e49-94cb-b8631abf4b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.152053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f54c3-c75d-4e49-94cb-b8631abf4b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.152341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f54c3-c75d-4e49-94cb-b8631abf4b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.152624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f54c3-c75d-4e49-94cb-b8631abf4b6f admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.153604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f54c3-c75d-4e49-94cb-b8631abf4b6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.155758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f54c3-c75d-4e49-94cb-b8631abf4b6f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.155914 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1089/2172] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:10:51.163361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b869b08-a59f-4d16-a354-598546bd9316 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.163651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b869b08-a59f-4d16-a354-598546bd9316 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.164531 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b869b08-a59f-4d16-a354-598546bd9316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.166052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:51.180752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1084/2173] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:10:51 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:10:51.184371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2b869b08-a59f-4d16-a354-598546bd9316 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:51.184616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f54c3-c75d-4e49-94cb-b8631abf4b6f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:51.185648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2b869b08-a59f-4d16-a354-598546bd9316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:51.186179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e80f54c3-c75d-4e49-94cb-b8631abf4b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:51.194944 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1090/2174] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:10:51.198234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b869b08-a59f-4d16-a354-598546bd9316 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:51.202188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f54c3-c75d-4e49-94cb-b8631abf4b6f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.207380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b869b08-a59f-4d16-a354-598546bd9316 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.207813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89486620-22ce-4425-8aec-d11a14b7ea55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.208626 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89486620-22ce-4425-8aec-d11a14b7ea55 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.208907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89486620-22ce-4425-8aec-d11a14b7ea55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.209180 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89486620-22ce-4425-8aec-d11a14b7ea55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.209945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89486620-22ce-4425-8aec-d11a14b7ea55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.211553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89486620-22ce-4425-8aec-d11a14b7ea55 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.212019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef426c-ce08-4241-bc70-2f6586fab83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.212664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef426c-ce08-4241-bc70-2f6586fab83c admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.212945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef426c-ce08-4241-bc70-2f6586fab83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.213217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef426c-ce08-4241-bc70-2f6586fab83c admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.213974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef426c-ce08-4241-bc70-2f6586fab83c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:10:51.214287 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef426c-ce08-4241-bc70-2f6586fab83c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.216140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef426c-ce08-4241-bc70-2f6586fab83c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.225305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1085/2175] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:10:51.226476 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89486620-22ce-4425-8aec-d11a14b7ea55 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:51.228236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1091/2176] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 10:10:51.232548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef426c-ce08-4241-bc70-2f6586fab83c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:51.236879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89486620-22ce-4425-8aec-d11a14b7ea55 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.241687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a924d5-2665-4dd2-bf5e-8def4711c727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.242086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef426c-ce08-4241-bc70-2f6586fab83c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.242606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a924d5-2665-4dd2-bf5e-8def4711c727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.243001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a924d5-2665-4dd2-bf5e-8def4711c727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.243287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a924d5-2665-4dd2-bf5e-8def4711c727 admin admin] PATH_INFO: `/v3/projects/59a4e51a19a949ddbfc1f72ef463810c/users/e74f4ce688f94affb80fec2a25743047/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.246535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dfca1-dfce-45e3-9d75-708984a0da03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.247283 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dfca1-dfce-45e3-9d75-708984a0da03 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.247705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dfca1-dfce-45e3-9d75-708984a0da03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.248188 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dfca1-dfce-45e3-9d75-708984a0da03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.248882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94dfca1-dfce-45e3-9d75-708984a0da03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.250827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94dfca1-dfce-45e3-9d75-708984a0da03 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.258716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a924d5-2665-4dd2-bf5e-8def4711c727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a4e51a19a949ddbfc1f72ef463810c, user_id=e74f4ce688f94affb80fec2a25743047, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.261440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a924d5-2665-4dd2-bf5e-8def4711c727 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.277663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1086/2177] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:51 2022] PUT /identity/v3/projects/59a4e51a19a949ddbfc1f72ef463810c/users/e74f4ce688f94affb80fec2a25743047/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:10:51.281761 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a924d5-2665-4dd2-bf5e-8def4711c727 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:51.282712 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1092/2178] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:10:51 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:10:51.286395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dfca1-dfce-45e3-9d75-708984a0da03 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:51.345152 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a924d5-2665-4dd2-bf5e-8def4711c727 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.347754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dfca1-dfce-45e3-9d75-708984a0da03 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.349863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a32cd-c419-4f2a-98c7-13b8422f7373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.350534 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a32cd-c419-4f2a-98c7-13b8422f7373 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.350828 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a32cd-c419-4f2a-98c7-13b8422f7373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.351091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a32cd-c419-4f2a-98c7-13b8422f7373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.351838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a32cd-c419-4f2a-98c7-13b8422f7373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.352721 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02392f-70f6-4700-8074-fa5f260e943c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.353389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02392f-70f6-4700-8074-fa5f260e943c admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.353492 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a32cd-c419-4f2a-98c7-13b8422f7373 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.353724 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02392f-70f6-4700-8074-fa5f260e943c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.354024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02392f-70f6-4700-8074-fa5f260e943c admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.354804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02392f-70f6-4700-8074-fa5f260e943c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.356919 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02392f-70f6-4700-8074-fa5f260e943c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.364785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1087/2179] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:10:51.368128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:51.368967 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a32cd-c419-4f2a-98c7-13b8422f7373 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:51.381527 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a32cd-c419-4f2a-98c7-13b8422f7373 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.387601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1093/2180] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:10:51 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:10:51.391340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05109597-0f8b-454d-9901-64b218613727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.391586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02392f-70f6-4700-8074-fa5f260e943c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:51.393141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05109597-0f8b-454d-9901-64b218613727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.393438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05109597-0f8b-454d-9901-64b218613727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.393766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05109597-0f8b-454d-9901-64b218613727 admin admin] PATH_INFO: `/v3/projects/59a4e51a19a949ddbfc1f72ef463810c/users/e74f4ce688f94affb80fec2a25743047/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.397361 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05109597-0f8b-454d-9901-64b218613727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a4e51a19a949ddbfc1f72ef463810c, user_id=e74f4ce688f94affb80fec2a25743047, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.401278 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05109597-0f8b-454d-9901-64b218613727 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.402836 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02392f-70f6-4700-8074-fa5f260e943c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.408936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6b4f0-6376-47aa-9a7c-f3f5848d315a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.410555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6b4f0-6376-47aa-9a7c-f3f5848d315a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.410555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6b4f0-6376-47aa-9a7c-f3f5848d315a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.410730 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6b4f0-6376-47aa-9a7c-f3f5848d315a admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.411541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6b4f0-6376-47aa-9a7c-f3f5848d315a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.413472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6b4f0-6376-47aa-9a7c-f3f5848d315a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.415805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1088/2181] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:51 2022] PUT /identity/v3/projects/59a4e51a19a949ddbfc1f72ef463810c/users/e74f4ce688f94affb80fec2a25743047/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:51.429772 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1094/2182] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:10:51.433776 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05109597-0f8b-454d-9901-64b218613727 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:51.524495 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05109597-0f8b-454d-9901-64b218613727 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.530167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad17a5-0900-4893-a08b-e4614adbc369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.530932 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad17a5-0900-4893-a08b-e4614adbc369 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.531221 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad17a5-0900-4893-a08b-e4614adbc369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.531503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad17a5-0900-4893-a08b-e4614adbc369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.532336 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad17a5-0900-4893-a08b-e4614adbc369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.534536 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad17a5-0900-4893-a08b-e4614adbc369 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.552646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1089/2183] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:10:51.554613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6b4f0-6376-47aa-9a7c-f3f5848d315a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:51.564452 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6b4f0-6376-47aa-9a7c-f3f5848d315a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.569551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685518c-c6e6-4edd-afb4-938715c23b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.570571 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685518c-c6e6-4edd-afb4-938715c23b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.570924 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685518c-c6e6-4edd-afb4-938715c23b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.571272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685518c-c6e6-4edd-afb4-938715c23b15 admin admin] PATH_INFO: `/v3/projects/c3069ceb69694ca5b45e09cfaecbd1eb/users/3644999b715d40db88cf06852158434d/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.588415 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685518c-c6e6-4edd-afb4-938715c23b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3069ceb69694ca5b45e09cfaecbd1eb, user_id=3644999b715d40db88cf06852158434d, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.591618 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685518c-c6e6-4edd-afb4-938715c23b15 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.615746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1095/2184] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:10:51 2022] PUT /identity/v3/projects/c3069ceb69694ca5b45e09cfaecbd1eb/users/3644999b715d40db88cf06852158434d/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:10:51.626444 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fea5b-0d4b-496e-a77d-ce1695a58255 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.626811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fea5b-0d4b-496e-a77d-ce1695a58255 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.627189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fea5b-0d4b-496e-a77d-ce1695a58255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.666393 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9d1fea5b-0d4b-496e-a77d-ce1695a58255 None None] MFA Rules not processed for user `3644999b715d40db88cf06852158434d`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:51.666393 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9d1fea5b-0d4b-496e-a77d-ce1695a58255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:51.707451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:51.753305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1090/2185] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:51.765332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685518c-c6e6-4edd-afb4-938715c23b15 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:51.767181 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f685518c-c6e6-4edd-afb4-938715c23b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:51.880431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685518c-c6e6-4edd-afb4-938715c23b15 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:51.888697 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072e427-9c6c-4676-b4dc-8b5e9de46c1f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef685a17c804e2bbfea1e07cfdf5cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:51.889378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072e427-9c6c-4676-b4dc-8b5e9de46c1f service glance] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:51.889685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072e427-9c6c-4676-b4dc-8b5e9de46c1f service glance] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:51.889969 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072e427-9c6c-4676-b4dc-8b5e9de46c1f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:51.921030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072e427-9c6c-4676-b4dc-8b5e9de46c1f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:51.922581 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072e427-9c6c-4676-b4dc-8b5e9de46c1f service glance] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:51.953850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1096/2186] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:52.496822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a968aff-56a1-4422-93ed-a83342bcb225 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:52.497475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a968aff-56a1-4422-93ed-a83342bcb225 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:52.497475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a968aff-56a1-4422-93ed-a83342bcb225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:52.528154 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7a968aff-56a1-4422-93ed-a83342bcb225 None None] MFA Rules not processed for user `e74f4ce688f94affb80fec2a25743047`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:52.539279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7a968aff-56a1-4422-93ed-a83342bcb225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:52.655390 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1091/2187] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:52 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:52.667269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072e427-9c6c-4676-b4dc-8b5e9de46c1f service glance] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:52.670009 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3072e427-9c6c-4676-b4dc-8b5e9de46c1f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:52.785923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072e427-9c6c-4676-b4dc-8b5e9de46c1f service glance] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:52.791563 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67959a9f-731e-42e0-b300-1ac1626ead09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:52.792640 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67959a9f-731e-42e0-b300-1ac1626ead09 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:52.793412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67959a9f-731e-42e0-b300-1ac1626ead09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:52.793733 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67959a9f-731e-42e0-b300-1ac1626ead09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:52.796139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-67959a9f-731e-42e0-b300-1ac1626ead09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:52.886809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67959a9f-731e-42e0-b300-1ac1626ead09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:52.888774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67959a9f-731e-42e0-b300-1ac1626ead09 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:52.899375 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1097/2188] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:54.311471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a968aff-56a1-4422-93ed-a83342bcb225 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:10:54.321334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a968aff-56a1-4422-93ed-a83342bcb225 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:10:54.325902 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d24d5-34e0-4049-9740-6bb7489b33cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:10:54.326439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d24d5-34e0-4049-9740-6bb7489b33cd service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:54.326789 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d24d5-34e0-4049-9740-6bb7489b33cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:54.327130 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d24d5-34e0-4049-9740-6bb7489b33cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:54.337056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d24d5-34e0-4049-9740-6bb7489b33cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:10:54.338670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d24d5-34e0-4049-9740-6bb7489b33cd service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:10:54.350231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1092/2189] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:10:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:10:54.415596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a274e7e-6801-4d79-a448-4aa896f42770 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:54.416107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a274e7e-6801-4d79-a448-4aa896f42770 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:54.416314 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a274e7e-6801-4d79-a448-4aa896f42770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:54.440928 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2a274e7e-6801-4d79-a448-4aa896f42770 None None] MFA Rules not processed for user `88d2351fc3b74ef5a530fded1be6b997`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:54.442182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2a274e7e-6801-4d79-a448-4aa896f42770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:54.454935 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:10:54.503590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1098/2190] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:10:58.220902 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fcef5-eccc-47a6-b1c1-ea163a9d69dc None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:10:58.221409 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fcef5-eccc-47a6-b1c1-ea163a9d69dc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:10:58.221501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fcef5-eccc-47a6-b1c1-ea163a9d69dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:10:58.241135 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-fd4fcef5-eccc-47a6-b1c1-ea163a9d69dc None None] MFA Rules not processed for user `fc9c26318d6c41f6b9455153f0d0e55b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:10:58.242592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fd4fcef5-eccc-47a6-b1c1-ea163a9d69dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:10:58.321651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1093/2191] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:10:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:04.494128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=17 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:08.314172 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=79 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:08.774442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a274e7e-6801-4d79-a448-4aa896f42770 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:08.933023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a274e7e-6801-4d79-a448-4aa896f42770 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:08.938733 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:08.939615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:08.939893 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:08.940162 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] PATH_INFO: `/v3/users/f1f5120a913c4a6db1a9eef0eebbeaa0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:08.941457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1f5120a913c4a6db1a9eef0eebbeaa0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:08.943380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:09.000203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1f5120a913c4a6db1a9eef0eebbeaa0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.000874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1f5120a913c4a6db1a9eef0eebbeaa0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.012280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1f5120a913c4a6db1a9eef0eebbeaa0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.026979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1f5120a913c4a6db1a9eef0eebbeaa0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.027781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1f5120a913c4a6db1a9eef0eebbeaa0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.042167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1099/2192] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:11:08 2022] DELETE /identity/v3/users/f1f5120a913c4a6db1a9eef0eebbeaa0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:11:09.191896 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03b2d5-00f4-4e05-8e18-41dfc6f38a48 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:09.192458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03b2d5-00f4-4e05-8e18-41dfc6f38a48 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:09.195860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03b2d5-00f4-4e05-8e18-41dfc6f38a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:09.231716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-8b03b2d5-00f4-4e05-8e18-41dfc6f38a48 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:09.232197 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:09.233153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8b03b2d5-00f4-4e05-8e18-41dfc6f38a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:09.234350 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:09.322014 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:09.336898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1094/2193] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:11:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:09.355253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03b2d5-00f4-4e05-8e18-41dfc6f38a48 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:09.357055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8b03b2d5-00f4-4e05-8e18-41dfc6f38a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:09.384783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176da8e-9b28-487d-9bbd-f2c7981d680f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:09.392874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:09.394803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:09.394940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:09.395377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] PATH_INFO: `/v3/projects/48aaf84d03a347bcb6ac42ca49b138cf` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:09.397522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48aaf84d03a347bcb6ac42ca49b138cf)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:09.401259 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:09.445890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03b2d5-00f4-4e05-8e18-41dfc6f38a48 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:09.452047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142ddb8-b979-4a8d-a82a-e7088f785405 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:09.453774 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142ddb8-b979-4a8d-a82a-e7088f785405 service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:09.454243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142ddb8-b979-4a8d-a82a-e7088f785405 service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:09.454842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142ddb8-b979-4a8d-a82a-e7088f785405 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:09.466564 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142ddb8-b979-4a8d-a82a-e7088f785405 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:09.469106 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142ddb8-b979-4a8d-a82a-e7088f785405 service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:09.477542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48aaf84d03a347bcb6ac42ca49b138cf', 'request_id': 'req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.479889 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1095/2194] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:11:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 10:11:09.486449 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48aaf84d03a347bcb6ac42ca49b138cf', 'request_id': 'req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.493869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48aaf84d03a347bcb6ac42ca49b138cf', 'request_id': 'req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.510006 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] The token cache is being invalidate because project 48aaf84d03a347bcb6ac42ca49b138cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:11:09.510294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.511087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.513686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1100/2195] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:11:09 2022] DELETE /identity/v3/projects/48aaf84d03a347bcb6ac42ca49b138cf => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:11:09.517088 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142ddb8-b979-4a8d-a82a-e7088f785405 service swift] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:09.529433 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142ddb8-b979-4a8d-a82a-e7088f785405 service swift] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:09.603905 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1c730-0f92-4f6e-b95e-5856b1a6a221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:09.604935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1c730-0f92-4f6e-b95e-5856b1a6a221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:09.605317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1c730-0f92-4f6e-b95e-5856b1a6a221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:09.605748 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1c730-0f92-4f6e-b95e-5856b1a6a221 admin admin] PATH_INFO: `/v3/users/e74f4ce688f94affb80fec2a25743047` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:09.607123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1c730-0f92-4f6e-b95e-5856b1a6a221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e74f4ce688f94affb80fec2a25743047)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:09.609267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1c730-0f92-4f6e-b95e-5856b1a6a221 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:09.622712 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:09.676470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fbc1c730-0f92-4f6e-b95e-5856b1a6a221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e74f4ce688f94affb80fec2a25743047'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.677236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fbc1c730-0f92-4f6e-b95e-5856b1a6a221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e74f4ce688f94affb80fec2a25743047'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.689539 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fbc1c730-0f92-4f6e-b95e-5856b1a6a221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e74f4ce688f94affb80fec2a25743047'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.704635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fbc1c730-0f92-4f6e-b95e-5856b1a6a221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e74f4ce688f94affb80fec2a25743047'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.705416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fbc1c730-0f92-4f6e-b95e-5856b1a6a221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e74f4ce688f94affb80fec2a25743047'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:09.721724 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1096/2196] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:11:09 2022] DELETE /identity/v3/users/e74f4ce688f94affb80fec2a25743047 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:11:10.029702 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:10.031424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:10.060234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2d169-b702-4a8d-907f-1ec65dc86ab9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:10.177642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315ce23-f116-4d73-852f-eb965bddd7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:10.178945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315ce23-f116-4d73-852f-eb965bddd7fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:10.179913 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315ce23-f116-4d73-852f-eb965bddd7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:10.179913 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315ce23-f116-4d73-852f-eb965bddd7fd admin admin] PATH_INFO: `/v3/projects/59a4e51a19a949ddbfc1f72ef463810c` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:10.181121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315ce23-f116-4d73-852f-eb965bddd7fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a4e51a19a949ddbfc1f72ef463810c)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:10.184213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315ce23-f116-4d73-852f-eb965bddd7fd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:10.290988 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b315ce23-f116-4d73-852f-eb965bddd7fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a4e51a19a949ddbfc1f72ef463810c', 'request_id': 'req-b315ce23-f116-4d73-852f-eb965bddd7fd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:10.300215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b315ce23-f116-4d73-852f-eb965bddd7fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a4e51a19a949ddbfc1f72ef463810c', 'request_id': 'req-b315ce23-f116-4d73-852f-eb965bddd7fd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:10.301546 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:10.315333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b315ce23-f116-4d73-852f-eb965bddd7fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a4e51a19a949ddbfc1f72ef463810c', 'request_id': 'req-b315ce23-f116-4d73-852f-eb965bddd7fd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:10.332226 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b315ce23-f116-4d73-852f-eb965bddd7fd admin admin] The token cache is being invalidate because project 59a4e51a19a949ddbfc1f72ef463810c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:11:10.332536 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b315ce23-f116-4d73-852f-eb965bddd7fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:10.333193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b315ce23-f116-4d73-852f-eb965bddd7fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:10.335560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1101/2197] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:11:10 2022] DELETE /identity/v3/projects/59a4e51a19a949ddbfc1f72ef463810c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:11:10.340812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177209c-6977-40fa-ae3c-dc3bd06147a3 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:10.341294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177209c-6977-40fa-ae3c-dc3bd06147a3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:10.341385 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177209c-6977-40fa-ae3c-dc3bd06147a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:10.393588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a177209c-6977-40fa-ae3c-dc3bd06147a3 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:10.395254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a177209c-6977-40fa-ae3c-dc3bd06147a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:10.454803 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1097/2198] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:10.464317 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b2b5d-f54f-4156-a511-36a97c745ab6 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:10.464317 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b2b5d-f54f-4156-a511-36a97c745ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:10.464317 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b2b5d-f54f-4156-a511-36a97c745ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:10.500789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-167b2b5d-f54f-4156-a511-36a97c745ab6 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:10.507954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-167b2b5d-f54f-4156-a511-36a97c745ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:10.526770 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1102/2199] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:10.528243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177209c-6977-40fa-ae3c-dc3bd06147a3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:10.548283 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177209c-6977-40fa-ae3c-dc3bd06147a3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:10.583782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15ed3f-6a97-4d36-a524-90bb36bc1b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:10.584854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15ed3f-6a97-4d36-a524-90bb36bc1b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:10.585289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15ed3f-6a97-4d36-a524-90bb36bc1b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:10.589529 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15ed3f-6a97-4d36-a524-90bb36bc1b44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:10.590618 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15ed3f-6a97-4d36-a524-90bb36bc1b44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:11:10.591108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15ed3f-6a97-4d36-a524-90bb36bc1b44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:10.593545 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15ed3f-6a97-4d36-a524-90bb36bc1b44 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:10.616215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1098/2200] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:11:10.617280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b2b5d-f54f-4156-a511-36a97c745ab6 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:10.632733 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b2b5d-f54f-4156-a511-36a97c745ab6 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:10.648867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f1901-9bf2-4142-b763-3aa15bbaad58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:10.650623 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f1901-9bf2-4142-b763-3aa15bbaad58 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:10.652407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f1901-9bf2-4142-b763-3aa15bbaad58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:10.653085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f1901-9bf2-4142-b763-3aa15bbaad58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:10.653985 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f1901-9bf2-4142-b763-3aa15bbaad58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:10.657416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f1901-9bf2-4142-b763-3aa15bbaad58 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:10.701337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15ed3f-6a97-4d36-a524-90bb36bc1b44 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:10.703332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1103/2201] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:11:10 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:11:10.717102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15ed3f-6a97-4d36-a524-90bb36bc1b44 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:10.795208 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:10.807090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134ec9f-1872-4678-996a-b714ea872d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:10.810712 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134ec9f-1872-4678-996a-b714ea872d5a admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:10.811368 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134ec9f-1872-4678-996a-b714ea872d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:10.811548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134ec9f-1872-4678-996a-b714ea872d5a admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:10.813586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134ec9f-1872-4678-996a-b714ea872d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:10.816974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134ec9f-1872-4678-996a-b714ea872d5a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:10.856522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1099/2202] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:11:10 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:11:10.860593 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f1901-9bf2-4142-b763-3aa15bbaad58 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:10.871636 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f1901-9bf2-4142-b763-3aa15bbaad58 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:10.877545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232799a-df57-4a45-b902-ad02070b1e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:10.881237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232799a-df57-4a45-b902-ad02070b1e0e admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:10.896975 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232799a-df57-4a45-b902-ad02070b1e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:10.897930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232799a-df57-4a45-b902-ad02070b1e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:10.898846 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232799a-df57-4a45-b902-ad02070b1e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:10.901289 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232799a-df57-4a45-b902-ad02070b1e0e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:10.927855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134ec9f-1872-4678-996a-b714ea872d5a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:10.928969 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1104/2203] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:11:10.942983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134ec9f-1872-4678-996a-b714ea872d5a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:10.958669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31317814-4628-4023-921b-059d4ec11d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:10.959730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31317814-4628-4023-921b-059d4ec11d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:10.960206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31317814-4628-4023-921b-059d4ec11d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:10.960695 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31317814-4628-4023-921b-059d4ec11d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:10.961649 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31317814-4628-4023-921b-059d4ec11d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:10.963727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31317814-4628-4023-921b-059d4ec11d42 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:10.982946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232799a-df57-4a45-b902-ad02070b1e0e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:10.990163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1100/2204] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:11:10.999299 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232799a-df57-4a45-b902-ad02070b1e0e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:11.005537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f17e9f-83df-4fd7-9b2e-9eac0c3dc49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:11.006729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f17e9f-83df-4fd7-9b2e-9eac0c3dc49a admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:11.007217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f17e9f-83df-4fd7-9b2e-9eac0c3dc49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:11.007595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f17e9f-83df-4fd7-9b2e-9eac0c3dc49a admin admin] PATH_INFO: `/v3/projects/5c2d128304c34e15ba283d3be7caec56/users/e746fe20c1224fcb819a3ac1a292708b/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:11.036075 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f17e9f-83df-4fd7-9b2e-9eac0c3dc49a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c2d128304c34e15ba283d3be7caec56, user_id=e746fe20c1224fcb819a3ac1a292708b, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:11.043213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f17e9f-83df-4fd7-9b2e-9eac0c3dc49a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:11.073321 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1105/2205] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:11:10 2022] PUT /identity/v3/projects/5c2d128304c34e15ba283d3be7caec56/users/e746fe20c1224fcb819a3ac1a292708b/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:11:11.085187 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939855c4-168d-4af3-ac09-adbfc952d278 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:11.085570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939855c4-168d-4af3-ac09-adbfc952d278 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:11.085916 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939855c4-168d-4af3-ac09-adbfc952d278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:11.137712 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-939855c4-168d-4af3-ac09-adbfc952d278 None None] MFA Rules not processed for user `e746fe20c1224fcb819a3ac1a292708b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:11.142047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-939855c4-168d-4af3-ac09-adbfc952d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:11.304693 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1101/2206] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:11.312933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f17e9f-83df-4fd7-9b2e-9eac0c3dc49a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:11.315897 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-41f17e9f-83df-4fd7-9b2e-9eac0c3dc49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:11.433369 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f17e9f-83df-4fd7-9b2e-9eac0c3dc49a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:11.438379 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba5004-32f1-4113-ae2a-50b97801a79d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef685a17c804e2bbfea1e07cfdf5cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:11.439367 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba5004-32f1-4113-ae2a-50b97801a79d service glance] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:11.439865 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba5004-32f1-4113-ae2a-50b97801a79d service glance] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:11.440332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba5004-32f1-4113-ae2a-50b97801a79d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:11.455187 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba5004-32f1-4113-ae2a-50b97801a79d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:11.456810 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba5004-32f1-4113-ae2a-50b97801a79d service glance] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:11.469534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1106/2207] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:11:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:11:21.292040 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=41,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:21.463954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=57,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:23.821059 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939855c4-168d-4af3-ac09-adbfc952d278 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:23.822585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-939855c4-168d-4af3-ac09-adbfc952d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:23.895576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939855c4-168d-4af3-ac09-adbfc952d278 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:23.900640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:23.901541 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:23.901858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:23.902147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] PATH_INFO: `/v3/users/3644999b715d40db88cf06852158434d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:23.903461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3644999b715d40db88cf06852158434d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:23.905412 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:23.965709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3644999b715d40db88cf06852158434d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:23.966329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3644999b715d40db88cf06852158434d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:23.976429 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3644999b715d40db88cf06852158434d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:23.991668 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3644999b715d40db88cf06852158434d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:23.992445 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3644999b715d40db88cf06852158434d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:24.006820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1102/2208] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:11:23 2022] DELETE /identity/v3/users/3644999b715d40db88cf06852158434d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:11:24.011283 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cbf16-2767-419f-903d-cbae4bdf9a06 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:24.011820 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cbf16-2767-419f-903d-cbae4bdf9a06 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:24.011820 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cbf16-2767-419f-903d-cbae4bdf9a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:24.030671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a27cbf16-2767-419f-903d-cbae4bdf9a06 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:24.032547 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a27cbf16-2767-419f-903d-cbae4bdf9a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:24.100779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1107/2209] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:24.111032 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:24.112845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:24.198896 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc177079-456d-48fb-9d42-59774515341b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:24.203167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5009380-006f-45ee-9b1a-e5647d941957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:24.203946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5009380-006f-45ee-9b1a-e5647d941957 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:24.204296 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5009380-006f-45ee-9b1a-e5647d941957 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:24.204680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5009380-006f-45ee-9b1a-e5647d941957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:24.215082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5009380-006f-45ee-9b1a-e5647d941957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:24.216595 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5009380-006f-45ee-9b1a-e5647d941957 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:24.226914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1103/2210] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:11:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:11:24.517241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cbf16-2767-419f-903d-cbae4bdf9a06 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:24.518761 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a27cbf16-2767-419f-903d-cbae4bdf9a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:24.531609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cbf16-2767-419f-903d-cbae4bdf9a06 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:24.537928 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299dcf62-75d7-4070-9087-c469a7fc7653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:24.539845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dcf62-75d7-4070-9087-c469a7fc7653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:24.540137 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dcf62-75d7-4070-9087-c469a7fc7653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:24.540419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dcf62-75d7-4070-9087-c469a7fc7653 admin admin] PATH_INFO: `/v3/projects/c3069ceb69694ca5b45e09cfaecbd1eb` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:24.542102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299dcf62-75d7-4070-9087-c469a7fc7653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3069ceb69694ca5b45e09cfaecbd1eb)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:24.544558 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299dcf62-75d7-4070-9087-c469a7fc7653 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:24.594933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-299dcf62-75d7-4070-9087-c469a7fc7653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3069ceb69694ca5b45e09cfaecbd1eb', 'request_id': 'req-299dcf62-75d7-4070-9087-c469a7fc7653'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:24.600839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-299dcf62-75d7-4070-9087-c469a7fc7653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3069ceb69694ca5b45e09cfaecbd1eb', 'request_id': 'req-299dcf62-75d7-4070-9087-c469a7fc7653'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:24.607003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-299dcf62-75d7-4070-9087-c469a7fc7653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3069ceb69694ca5b45e09cfaecbd1eb', 'request_id': 'req-299dcf62-75d7-4070-9087-c469a7fc7653'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:24.623046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-299dcf62-75d7-4070-9087-c469a7fc7653 admin admin] The token cache is being invalidate because project c3069ceb69694ca5b45e09cfaecbd1eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:11:24.623307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-299dcf62-75d7-4070-9087-c469a7fc7653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:24.623994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-299dcf62-75d7-4070-9087-c469a7fc7653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:24.625555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1108/2211] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:11:24 2022] DELETE /identity/v3/projects/c3069ceb69694ca5b45e09cfaecbd1eb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:11:24.630740 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81955-623e-4892-abe7-ef3b80523b5d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:24.631060 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81955-623e-4892-abe7-ef3b80523b5d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:24.631346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81955-623e-4892-abe7-ef3b80523b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:24.634993 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:24.661146 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-64e81955-623e-4892-abe7-ef3b80523b5d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:24.662412 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-64e81955-623e-4892-abe7-ef3b80523b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:24.723190 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1104/2212] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:24.732317 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f1a71-2183-46c8-862a-93017c37d71f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:24.732631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f1a71-2183-46c8-862a-93017c37d71f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:24.733050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f1a71-2183-46c8-862a-93017c37d71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:24.753545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-cb5f1a71-2183-46c8-862a-93017c37d71f None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:24.754801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-cb5f1a71-2183-46c8-862a-93017c37d71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:24.763788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1109/2213] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:24.768438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e81955-623e-4892-abe7-ef3b80523b5d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:24.778424 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e81955-623e-4892-abe7-ef3b80523b5d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:24.783576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3634d4-6375-493e-80f4-495049ae6684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:24.784237 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3634d4-6375-493e-80f4-495049ae6684 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:24.784524 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3634d4-6375-493e-80f4-495049ae6684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:24.784836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3634d4-6375-493e-80f4-495049ae6684 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:24.785559 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3634d4-6375-493e-80f4-495049ae6684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:11:24.785903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3634d4-6375-493e-80f4-495049ae6684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:24.787523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3634d4-6375-493e-80f4-495049ae6684 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:24.802487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1105/2214] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:11:24.807300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f1a71-2183-46c8-862a-93017c37d71f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:24.816955 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f1a71-2183-46c8-862a-93017c37d71f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:24.822049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f444a-4c81-4237-b55d-24c495359950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:24.822686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f444a-4c81-4237-b55d-24c495359950 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:24.822970 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f444a-4c81-4237-b55d-24c495359950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:24.823246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f444a-4c81-4237-b55d-24c495359950 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:24.823940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f444a-4c81-4237-b55d-24c495359950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:24.825897 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f444a-4c81-4237-b55d-24c495359950 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:24.856979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1110/2215] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:11:24 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:11:24.862205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3634d4-6375-493e-80f4-495049ae6684 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:24.873759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3634d4-6375-493e-80f4-495049ae6684 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:24.937807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca7952-ad17-4099-b49c-82fd9d3414aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:24.938497 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca7952-ad17-4099-b49c-82fd9d3414aa admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:24.938783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca7952-ad17-4099-b49c-82fd9d3414aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:24.939349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca7952-ad17-4099-b49c-82fd9d3414aa admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:24.940337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca7952-ad17-4099-b49c-82fd9d3414aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:24.942337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca7952-ad17-4099-b49c-82fd9d3414aa admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:24.966976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1106/2216] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:11:24 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:11:24.972175 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f444a-4c81-4237-b55d-24c495359950 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:24.982902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f444a-4c81-4237-b55d-24c495359950 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:24.988732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57636b-0d69-479d-bd25-338b990668e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:24.989392 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57636b-0d69-479d-bd25-338b990668e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:24.989706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57636b-0d69-479d-bd25-338b990668e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:24.989983 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57636b-0d69-479d-bd25-338b990668e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:24.990711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57636b-0d69-479d-bd25-338b990668e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:24.992331 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57636b-0d69-479d-bd25-338b990668e5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:25.002921 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1111/2217] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:11:25.007701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca7952-ad17-4099-b49c-82fd9d3414aa admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:25.024261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca7952-ad17-4099-b49c-82fd9d3414aa admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:25.033372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53f138-2918-4f1b-b2b6-56869fa8bb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:25.034105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53f138-2918-4f1b-b2b6-56869fa8bb26 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:25.034388 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53f138-2918-4f1b-b2b6-56869fa8bb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:25.034664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53f138-2918-4f1b-b2b6-56869fa8bb26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:25.035406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53f138-2918-4f1b-b2b6-56869fa8bb26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:25.037031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53f138-2918-4f1b-b2b6-56869fa8bb26 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:25.047621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1107/2218] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:11:25.053576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57636b-0d69-479d-bd25-338b990668e5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:25.066910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57636b-0d69-479d-bd25-338b990668e5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:25.072758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33807b-613e-4f44-bf70-13a7f5a25b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:25.073654 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33807b-613e-4f44-bf70-13a7f5a25b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:25.073960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33807b-613e-4f44-bf70-13a7f5a25b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:25.074247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33807b-613e-4f44-bf70-13a7f5a25b11 admin admin] PATH_INFO: `/v3/projects/f8d7bb97464d4807a6b7bbac9a41dc67/users/f7c009e1bcd74151ad3fa95c68916329/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:25.093061 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33807b-613e-4f44-bf70-13a7f5a25b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d7bb97464d4807a6b7bbac9a41dc67, user_id=f7c009e1bcd74151ad3fa95c68916329, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:25.096459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33807b-613e-4f44-bf70-13a7f5a25b11 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:25.112869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1112/2219] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:11:25 2022] PUT /identity/v3/projects/f8d7bb97464d4807a6b7bbac9a41dc67/users/f7c009e1bcd74151ad3fa95c68916329/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:11:25.118627 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2dfd7-ff1f-4df4-a4ad-f84711cf2233 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:25.119044 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2dfd7-ff1f-4df4-a4ad-f84711cf2233 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:25.119661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2dfd7-ff1f-4df4-a4ad-f84711cf2233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:25.147603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d7e2dfd7-ff1f-4df4-a4ad-f84711cf2233 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:25.148907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d7e2dfd7-ff1f-4df4-a4ad-f84711cf2233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:25.199891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:25.211297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1108/2220] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:25.221098 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33807b-613e-4f44-bf70-13a7f5a25b11 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:25.222871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4c33807b-613e-4f44-bf70-13a7f5a25b11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:25.244734 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:25.283626 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33807b-613e-4f44-bf70-13a7f5a25b11 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:25.287572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62c0f7-f9d0-45a3-8494-098e9166a59f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:25.288230 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62c0f7-f9d0-45a3-8494-098e9166a59f service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:25.288514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62c0f7-f9d0-45a3-8494-098e9166a59f service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:25.288793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62c0f7-f9d0-45a3-8494-098e9166a59f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:25.299678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62c0f7-f9d0-45a3-8494-098e9166a59f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:25.302044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62c0f7-f9d0-45a3-8494-098e9166a59f service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:25.313726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1113/2221] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:11:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:11:25.904416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103fc33-7871-473b-878c-fff16de56df7 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:25.904416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103fc33-7871-473b-878c-fff16de56df7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:25.905380 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103fc33-7871-473b-878c-fff16de56df7 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:25.907422 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1109/2222] 10.222.0.79 () {54 vars in 921 bytes} [Mon Jul 4 10:11:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:11:25.914973 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab51ad-3e4f-40ec-851d-42282d053fe6 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:25.915274 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab51ad-3e4f-40ec-851d-42282d053fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:25.915548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab51ad-3e4f-40ec-851d-42282d053fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:26.000416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-09ab51ad-3e4f-40ec-851d-42282d053fe6 None None] MFA Rules not processed for user `ffc2409cb7fc4373a74ddfb48d501d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:26.002026 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-09ab51ad-3e4f-40ec-851d-42282d053fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:26.106358 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1114/2223] 10.222.0.79 () {58 vars in 1018 bytes} [Mon Jul 4 10:11:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:26.118000 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103fc33-7871-473b-878c-fff16de56df7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:26.120310 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8103fc33-7871-473b-878c-fff16de56df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:26.193047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103fc33-7871-473b-878c-fff16de56df7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:26.198867 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151b098-1e8a-4a9d-94c7-285d865f3106 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef685a17c804e2bbfea1e07cfdf5cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:26.199660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151b098-1e8a-4a9d-94c7-285d865f3106 service glance] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:26.200114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151b098-1e8a-4a9d-94c7-285d865f3106 service glance] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:26.200626 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151b098-1e8a-4a9d-94c7-285d865f3106 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:26.212782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151b098-1e8a-4a9d-94c7-285d865f3106 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:26.214402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151b098-1e8a-4a9d-94c7-285d865f3106 service glance] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:26.228591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1110/2224] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:11:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:11:26.306759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be9f18-1d39-447f-be98-f3c1e0da88c9 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:26.307520 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be9f18-1d39-447f-be98-f3c1e0da88c9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:26.307993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be9f18-1d39-447f-be98-f3c1e0da88c9 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:26.309535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1115/2225] 10.222.0.79 () {54 vars in 921 bytes} [Mon Jul 4 10:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:11:26.319151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c74b97-2293-4d99-9df6-2b0d63b89a3a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:26.319677 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c74b97-2293-4d99-9df6-2b0d63b89a3a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:26.320129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c74b97-2293-4d99-9df6-2b0d63b89a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:26.340291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-29c74b97-2293-4d99-9df6-2b0d63b89a3a None None] MFA Rules not processed for user `3a1f11a4b8554a98b7626a82f01888c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:26.341515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-29c74b97-2293-4d99-9df6-2b0d63b89a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:26.350307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1111/2226] 10.222.0.79 () {58 vars in 1018 bytes} [Mon Jul 4 10:11:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:26.364048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be9f18-1d39-447f-be98-f3c1e0da88c9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:26.378147 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be9f18-1d39-447f-be98-f3c1e0da88c9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:26.381889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e81d42-9887-4fe9-aae9-c76edf9babce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:26.382545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e81d42-9887-4fe9-aae9-c76edf9babce service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:26.382832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e81d42-9887-4fe9-aae9-c76edf9babce service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:26.383109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e81d42-9887-4fe9-aae9-c76edf9babce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:26.391164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e81d42-9887-4fe9-aae9-c76edf9babce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:26.393341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e81d42-9887-4fe9-aae9-c76edf9babce service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:26.402085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1116/2227] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:11:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:11:26.495796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bee0c9-a86b-42bd-81ec-9df2345df71b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:26.496232 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bee0c9-a86b-42bd-81ec-9df2345df71b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:26.496518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bee0c9-a86b-42bd-81ec-9df2345df71b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:26.498214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1112/2228] 10.222.0.79 () {54 vars in 921 bytes} [Mon Jul 4 10:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 10:11:26.504210 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7258ec6-43ff-49ef-8880-5a6c706e4390 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:26.504499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7258ec6-43ff-49ef-8880-5a6c706e4390 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:26.504777 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7258ec6-43ff-49ef-8880-5a6c706e4390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:26.524833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b7258ec6-43ff-49ef-8880-5a6c706e4390 None None] MFA Rules not processed for user `ffc2409cb7fc4373a74ddfb48d501d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:26.526123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b7258ec6-43ff-49ef-8880-5a6c706e4390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:26.537044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1117/2229] 10.222.0.79 () {58 vars in 1018 bytes} [Mon Jul 4 10:11:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:26.547732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bee0c9-a86b-42bd-81ec-9df2345df71b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:26.559175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bee0c9-a86b-42bd-81ec-9df2345df71b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:26.563540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf96de-396e-4fc9-94ce-5330e5fd8d77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:26.564295 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf96de-396e-4fc9-94ce-5330e5fd8d77 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:26.565110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf96de-396e-4fc9-94ce-5330e5fd8d77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:26.565110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf96de-396e-4fc9-94ce-5330e5fd8d77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:26.577238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf96de-396e-4fc9-94ce-5330e5fd8d77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:26.579105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf96de-396e-4fc9-94ce-5330e5fd8d77 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:26.594085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1113/2230] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:11:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:11:29.780370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ddb6c-3f7f-4cdc-99d5-ded8a26df3f8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:29.781371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ddb6c-3f7f-4cdc-99d5-ded8a26df3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:29.781913 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ddb6c-3f7f-4cdc-99d5-ded8a26df3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:29.831165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a71ddb6c-3f7f-4cdc-99d5-ded8a26df3f8 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:29.832911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a71ddb6c-3f7f-4cdc-99d5-ded8a26df3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:29.935119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:29.965290 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1118/2231] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:11:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:29.995905 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf96de-396e-4fc9-94ce-5330e5fd8d77 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:29.998677 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-30bf96de-396e-4fc9-94ce-5330e5fd8d77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:30.097987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf96de-396e-4fc9-94ce-5330e5fd8d77 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:30.105860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a0c2f-f71e-4d87-96d2-4138ef2ab5b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:30.106551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a0c2f-f71e-4d87-96d2-4138ef2ab5b1 service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:30.106836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a0c2f-f71e-4d87-96d2-4138ef2ab5b1 service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:30.107113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a0c2f-f71e-4d87-96d2-4138ef2ab5b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:30.117920 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a0c2f-f71e-4d87-96d2-4138ef2ab5b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:30.120156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a0c2f-f71e-4d87-96d2-4138ef2ab5b1 service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:30.137385 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1114/2232] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:11:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 10:11:30.486876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f1caa-7011-4dba-9366-9a54ed954d86 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:30.487397 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f1caa-7011-4dba-9366-9a54ed954d86 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:30.487883 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f1caa-7011-4dba-9366-9a54ed954d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:30.536075 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-642f1caa-7011-4dba-9366-9a54ed954d86 None None] MFA Rules not processed for user `f7c009e1bcd74151ad3fa95c68916329`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:30.537735 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-642f1caa-7011-4dba-9366-9a54ed954d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:30.696291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a0c2f-f71e-4d87-96d2-4138ef2ab5b1 service swift] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:30.697117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1119/2233] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:30.708263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a0c2f-f71e-4d87-96d2-4138ef2ab5b1 service swift] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:30.714044 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff81d7-b7eb-46ad-a6b2-8dfa1c00a493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:30.714978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff81d7-b7eb-46ad-a6b2-8dfa1c00a493 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:30.715346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff81d7-b7eb-46ad-a6b2-8dfa1c00a493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:30.715704 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff81d7-b7eb-46ad-a6b2-8dfa1c00a493 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:30.716523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff81d7-b7eb-46ad-a6b2-8dfa1c00a493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:30.719309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff81d7-b7eb-46ad-a6b2-8dfa1c00a493 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:30.804565 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1115/2234] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:11:30 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:11:30.806686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f1caa-7011-4dba-9366-9a54ed954d86 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:30.820705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f1caa-7011-4dba-9366-9a54ed954d86 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:30.916183 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1042ea-c074-4fd7-bff7-465144d44931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:30.916981 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1042ea-c074-4fd7-bff7-465144d44931 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:30.917335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1042ea-c074-4fd7-bff7-465144d44931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:30.917943 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1042ea-c074-4fd7-bff7-465144d44931 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:30.918725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1042ea-c074-4fd7-bff7-465144d44931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:30.921183 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1042ea-c074-4fd7-bff7-465144d44931 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:30.959364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1120/2235] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:11:30 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:11:30.963753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff81d7-b7eb-46ad-a6b2-8dfa1c00a493 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:30.978300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff81d7-b7eb-46ad-a6b2-8dfa1c00a493 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:30.984449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d92cf8-5d3a-4112-81fe-6dc5f066df1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:30.988463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d92cf8-5d3a-4112-81fe-6dc5f066df1b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:30.988742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d92cf8-5d3a-4112-81fe-6dc5f066df1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:30.989071 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d92cf8-5d3a-4112-81fe-6dc5f066df1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:30.989948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d92cf8-5d3a-4112-81fe-6dc5f066df1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:30.991624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d92cf8-5d3a-4112-81fe-6dc5f066df1b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:31.004173 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1116/2236] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:11:31.008535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1042ea-c074-4fd7-bff7-465144d44931 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:31.025363 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1042ea-c074-4fd7-bff7-465144d44931 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:31.033654 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12c25d-5f65-4b0e-a79b-24d58aafc272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:31.036567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12c25d-5f65-4b0e-a79b-24d58aafc272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:31.037752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12c25d-5f65-4b0e-a79b-24d58aafc272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:31.038176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12c25d-5f65-4b0e-a79b-24d58aafc272 admin admin] PATH_INFO: `/v3/projects/1438c8b530734ec5871e8a267fd2baf5/users/35af782693ed437d88f25d272b9be9e7/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:31.055563 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12c25d-5f65-4b0e-a79b-24d58aafc272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1438c8b530734ec5871e8a267fd2baf5, user_id=35af782693ed437d88f25d272b9be9e7, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:31.058368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12c25d-5f65-4b0e-a79b-24d58aafc272 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:31.075393 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1121/2237] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:11:31 2022] PUT /identity/v3/projects/1438c8b530734ec5871e8a267fd2baf5/users/35af782693ed437d88f25d272b9be9e7/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:11:31.079728 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d92cf8-5d3a-4112-81fe-6dc5f066df1b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:31.090099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d92cf8-5d3a-4112-81fe-6dc5f066df1b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:31.132128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:31.165408 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84121c12-1c55-495a-bd0f-78804aa3e31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:31.166202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84121c12-1c55-495a-bd0f-78804aa3e31e admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:31.166664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84121c12-1c55-495a-bd0f-78804aa3e31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:31.167002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84121c12-1c55-495a-bd0f-78804aa3e31e admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:31.167835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84121c12-1c55-495a-bd0f-78804aa3e31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:31.169699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84121c12-1c55-495a-bd0f-78804aa3e31e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:31.180922 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1117/2238] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:11:31.185136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12c25d-5f65-4b0e-a79b-24d58aafc272 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:31.196335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12c25d-5f65-4b0e-a79b-24d58aafc272 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:31.204105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7456ba7-234b-4de8-b7f7-c8c612fbbf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:31.205297 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7456ba7-234b-4de8-b7f7-c8c612fbbf00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:31.205606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7456ba7-234b-4de8-b7f7-c8c612fbbf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:31.205920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7456ba7-234b-4de8-b7f7-c8c612fbbf00 admin admin] PATH_INFO: `/v3/projects/1438c8b530734ec5871e8a267fd2baf5/users/35af782693ed437d88f25d272b9be9e7/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:31.208107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7456ba7-234b-4de8-b7f7-c8c612fbbf00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1438c8b530734ec5871e8a267fd2baf5, user_id=35af782693ed437d88f25d272b9be9e7, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:31.210859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7456ba7-234b-4de8-b7f7-c8c612fbbf00 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:31.223249 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1122/2239] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:11:31 2022] PUT /identity/v3/projects/1438c8b530734ec5871e8a267fd2baf5/users/35af782693ed437d88f25d272b9be9e7/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:11:31.904328 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84121c12-1c55-495a-bd0f-78804aa3e31e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:31.907300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-84121c12-1c55-495a-bd0f-78804aa3e31e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:31.940426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84121c12-1c55-495a-bd0f-78804aa3e31e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:32.085580 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:32.088850 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:32.090006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:32.090349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] PATH_INFO: `/v3/users/e746fe20c1224fcb819a3ac1a292708b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:32.096266 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e746fe20c1224fcb819a3ac1a292708b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:32.098337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:32.221955 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e746fe20c1224fcb819a3ac1a292708b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:32.225292 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e746fe20c1224fcb819a3ac1a292708b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:32.238268 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e746fe20c1224fcb819a3ac1a292708b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:32.266924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e746fe20c1224fcb819a3ac1a292708b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:32.268684 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e746fe20c1224fcb819a3ac1a292708b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:32.296258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1118/2240] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:11:31 2022] DELETE /identity/v3/users/e746fe20c1224fcb819a3ac1a292708b => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:11:32.301304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed484c2-e436-4469-abac-e0929f16e745 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:32.301843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed484c2-e436-4469-abac-e0929f16e745 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:32.302138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed484c2-e436-4469-abac-e0929f16e745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:32.342965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-fed484c2-e436-4469-abac-e0929f16e745 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:32.347150 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fed484c2-e436-4469-abac-e0929f16e745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:32.411498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:32.475342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1123/2241] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:32.488205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:32.489685 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:32.648269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce913d-fb64-4925-a5aa-5ff2a28b6b73 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:32.662553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ecd57-a436-432a-9eaf-dad5daad7400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:32.663314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ecd57-a436-432a-9eaf-dad5daad7400 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:32.663600 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ecd57-a436-432a-9eaf-dad5daad7400 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:32.664119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ecd57-a436-432a-9eaf-dad5daad7400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:32.687134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ecd57-a436-432a-9eaf-dad5daad7400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:32.693314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ecd57-a436-432a-9eaf-dad5daad7400 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:32.704813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:32.737592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1119/2242] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:11:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:11:33.178224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed484c2-e436-4469-abac-e0929f16e745 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:33.179907 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fed484c2-e436-4469-abac-e0929f16e745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:33.207494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed484c2-e436-4469-abac-e0929f16e745 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:33.224133 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914cb8d-160b-4500-bbdc-81f5ee2620a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:33.225317 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914cb8d-160b-4500-bbdc-81f5ee2620a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:33.241851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914cb8d-160b-4500-bbdc-81f5ee2620a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:33.241851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914cb8d-160b-4500-bbdc-81f5ee2620a7 admin admin] PATH_INFO: `/v3/projects/5c2d128304c34e15ba283d3be7caec56` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:33.241851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914cb8d-160b-4500-bbdc-81f5ee2620a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c2d128304c34e15ba283d3be7caec56)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:33.241851 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914cb8d-160b-4500-bbdc-81f5ee2620a7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:33.341081 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c914cb8d-160b-4500-bbdc-81f5ee2620a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c2d128304c34e15ba283d3be7caec56', 'request_id': 'req-c914cb8d-160b-4500-bbdc-81f5ee2620a7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:33.373984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c914cb8d-160b-4500-bbdc-81f5ee2620a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c2d128304c34e15ba283d3be7caec56', 'request_id': 'req-c914cb8d-160b-4500-bbdc-81f5ee2620a7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:33.389915 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c914cb8d-160b-4500-bbdc-81f5ee2620a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c2d128304c34e15ba283d3be7caec56', 'request_id': 'req-c914cb8d-160b-4500-bbdc-81f5ee2620a7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:33.426144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c914cb8d-160b-4500-bbdc-81f5ee2620a7 admin admin] The token cache is being invalidate because project 5c2d128304c34e15ba283d3be7caec56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:11:33.426710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c914cb8d-160b-4500-bbdc-81f5ee2620a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:33.430019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c914cb8d-160b-4500-bbdc-81f5ee2620a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:33.435366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1124/2243] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:11:33 2022] DELETE /identity/v3/projects/5c2d128304c34e15ba283d3be7caec56 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:11:33.442083 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d765e3-6eaf-4edd-81d8-c8ab8fa2f554 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:33.447436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d765e3-6eaf-4edd-81d8-c8ab8fa2f554 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:33.447676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d765e3-6eaf-4edd-81d8-c8ab8fa2f554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:33.483910 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e8d765e3-6eaf-4edd-81d8-c8ab8fa2f554 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:33.485308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e8d765e3-6eaf-4edd-81d8-c8ab8fa2f554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:33.681040 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1120/2244] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:33.762544 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed74453-7ec3-461e-ab81-d3ca6633978a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:33.763034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed74453-7ec3-461e-ab81-d3ca6633978a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:33.763141 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed74453-7ec3-461e-ab81-d3ca6633978a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:33.807878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2ed74453-7ec3-461e-ab81-d3ca6633978a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:33.809223 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2ed74453-7ec3-461e-ab81-d3ca6633978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:33.850834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1125/2245] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:33.852611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d765e3-6eaf-4edd-81d8-c8ab8fa2f554 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:33.871036 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d765e3-6eaf-4edd-81d8-c8ab8fa2f554 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:33.893209 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da71c1-b187-474b-940d-cacd487cda6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:33.893973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da71c1-b187-474b-940d-cacd487cda6d admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:33.894259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da71c1-b187-474b-940d-cacd487cda6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:33.894535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da71c1-b187-474b-940d-cacd487cda6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:33.901693 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da71c1-b187-474b-940d-cacd487cda6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:11:33.902128 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da71c1-b187-474b-940d-cacd487cda6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:33.910587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da71c1-b187-474b-940d-cacd487cda6d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:33.971646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed74453-7ec3-461e-ab81-d3ca6633978a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:33.972035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1121/2246] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:11:33.988771 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed74453-7ec3-461e-ab81-d3ca6633978a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:34.008679 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c41476-ea47-4009-8ee7-098420790e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:34.009377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c41476-ea47-4009-8ee7-098420790e8a admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:34.009700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c41476-ea47-4009-8ee7-098420790e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:34.009954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c41476-ea47-4009-8ee7-098420790e8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:34.010653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c41476-ea47-4009-8ee7-098420790e8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:34.012836 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c41476-ea47-4009-8ee7-098420790e8a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:34.112486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1126/2247] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:11:33 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:11:34.113394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da71c1-b187-474b-940d-cacd487cda6d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:34.142839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da71c1-b187-474b-940d-cacd487cda6d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:34.365813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f4d14-cb98-4321-8bff-00ef1aae1d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:34.366698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f4d14-cb98-4321-8bff-00ef1aae1d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:34.366998 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f4d14-cb98-4321-8bff-00ef1aae1d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:34.367287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f4d14-cb98-4321-8bff-00ef1aae1d73 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:34.368021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f4d14-cb98-4321-8bff-00ef1aae1d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:34.373178 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f4d14-cb98-4321-8bff-00ef1aae1d73 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:34.457672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c41476-ea47-4009-8ee7-098420790e8a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:34.458208 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1122/2248] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:11:34 2022] POST /identity/v3/users => generated 288 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:11:34.470034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c41476-ea47-4009-8ee7-098420790e8a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:34.494365 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c984e2-e100-4a93-9939-c84778bfae98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:34.495068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c984e2-e100-4a93-9939-c84778bfae98 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:34.495361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c984e2-e100-4a93-9939-c84778bfae98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:34.495638 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c984e2-e100-4a93-9939-c84778bfae98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:34.496436 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c984e2-e100-4a93-9939-c84778bfae98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:34.498176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c984e2-e100-4a93-9939-c84778bfae98 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:34.538820 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1127/2249] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:11:34.540164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f4d14-cb98-4321-8bff-00ef1aae1d73 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:34.557257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f4d14-cb98-4321-8bff-00ef1aae1d73 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:34.570820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca5f57-0b62-4402-aad0-375473a78f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:34.571696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca5f57-0b62-4402-aad0-375473a78f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:34.572151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca5f57-0b62-4402-aad0-375473a78f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:34.572579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca5f57-0b62-4402-aad0-375473a78f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:34.573474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca5f57-0b62-4402-aad0-375473a78f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:34.575321 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca5f57-0b62-4402-aad0-375473a78f9a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:34.635834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1123/2250] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:11:34.639231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c984e2-e100-4a93-9939-c84778bfae98 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:34.664010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c984e2-e100-4a93-9939-c84778bfae98 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:34.684691 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45622a-8e01-4d3b-a096-918dbee97690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:34.685664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45622a-8e01-4d3b-a096-918dbee97690 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:34.685937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45622a-8e01-4d3b-a096-918dbee97690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:34.686913 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45622a-8e01-4d3b-a096-918dbee97690 admin admin] PATH_INFO: `/v3/projects/3d0dea7ee2c840cb8ff7b4bd243c6da1/users/a98455f8df4c425c9b65eed8bba4b4c8/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:34.727762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45622a-8e01-4d3b-a096-918dbee97690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d0dea7ee2c840cb8ff7b4bd243c6da1, user_id=a98455f8df4c425c9b65eed8bba4b4c8, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:34.731000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45622a-8e01-4d3b-a096-918dbee97690 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:34.775962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1128/2251] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:11:34 2022] PUT /identity/v3/projects/3d0dea7ee2c840cb8ff7b4bd243c6da1/users/a98455f8df4c425c9b65eed8bba4b4c8/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:11:34.777663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbc380-5745-4883-96cd-c759d5ce6ef4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:34.778006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbc380-5745-4883-96cd-c759d5ce6ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:34.778292 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbc380-5745-4883-96cd-c759d5ce6ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:34.861833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9cfbc380-5745-4883-96cd-c759d5ce6ef4 None None] MFA Rules not processed for user `a98455f8df4c425c9b65eed8bba4b4c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:34.863109 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9cfbc380-5745-4883-96cd-c759d5ce6ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:35.039120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:35.086652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1124/2252] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:11:35.111385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45622a-8e01-4d3b-a096-918dbee97690 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:35.115991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1e45622a-8e01-4d3b-a096-918dbee97690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:35.238505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:35.294944 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45622a-8e01-4d3b-a096-918dbee97690 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:35.309633 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b419c7e-de75-49ae-a135-76855cefff12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:35.310320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b419c7e-de75-49ae-a135-76855cefff12 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:35.310609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b419c7e-de75-49ae-a135-76855cefff12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:35.310893 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b419c7e-de75-49ae-a135-76855cefff12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:35.336825 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b419c7e-de75-49ae-a135-76855cefff12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:35.340364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b419c7e-de75-49ae-a135-76855cefff12 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:35.362123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1129/2253] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:11:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:11:36.119545 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbc380-5745-4883-96cd-c759d5ce6ef4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:36.121588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9cfbc380-5745-4883-96cd-c759d5ce6ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:36.363871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbc380-5745-4883-96cd-c759d5ce6ef4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:36.379118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266faa8b-1583-46c0-9ac6-828058f8eb7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:36.379829 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266faa8b-1583-46c0-9ac6-828058f8eb7a service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:36.380216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266faa8b-1583-46c0-9ac6-828058f8eb7a service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:36.380600 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266faa8b-1583-46c0-9ac6-828058f8eb7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:36.384104 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-266faa8b-1583-46c0-9ac6-828058f8eb7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:36.412690 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266faa8b-1583-46c0-9ac6-828058f8eb7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:36.423544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266faa8b-1583-46c0-9ac6-828058f8eb7a service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:36.449914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1125/2254] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:11:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:11:39.361545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b419c7e-de75-49ae-a135-76855cefff12 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:39.537490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b419c7e-de75-49ae-a135-76855cefff12 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:39.538236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bc370-b013-43c7-b08b-f8487d6e6382 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:39.539097 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bc370-b013-43c7-b08b-f8487d6e6382 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:39.539560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bc370-b013-43c7-b08b-f8487d6e6382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:39.543202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b90690-3a78-4c80-8a48-331b6c28006b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:39.544002 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b90690-3a78-4c80-8a48-331b6c28006b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:39.544349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b90690-3a78-4c80-8a48-331b6c28006b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:39.544695 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b90690-3a78-4c80-8a48-331b6c28006b admin admin] PATH_INFO: `/v3/users/a98455f8df4c425c9b65eed8bba4b4c8` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:39.546040 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b90690-3a78-4c80-8a48-331b6c28006b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a98455f8df4c425c9b65eed8bba4b4c8)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:39.554303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b90690-3a78-4c80-8a48-331b6c28006b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:39.587893 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f60bc370-b013-43c7-b08b-f8487d6e6382 None None] MFA Rules not processed for user `35af782693ed437d88f25d272b9be9e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:39.589189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f60bc370-b013-43c7-b08b-f8487d6e6382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:39.657337 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-76b90690-3a78-4c80-8a48-331b6c28006b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a98455f8df4c425c9b65eed8bba4b4c8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:39.658293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-76b90690-3a78-4c80-8a48-331b6c28006b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a98455f8df4c425c9b65eed8bba4b4c8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:39.682561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-76b90690-3a78-4c80-8a48-331b6c28006b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a98455f8df4c425c9b65eed8bba4b4c8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:39.699514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-76b90690-3a78-4c80-8a48-331b6c28006b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a98455f8df4c425c9b65eed8bba4b4c8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:39.700385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-76b90690-3a78-4c80-8a48-331b6c28006b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a98455f8df4c425c9b65eed8bba4b4c8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:39.719866 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1130/2255] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:11:39 2022] DELETE /identity/v3/users/a98455f8df4c425c9b65eed8bba4b4c8 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:11:39.725720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31925b34-f8f2-4b0f-aea5-d2857b22bad3 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:39.727456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31925b34-f8f2-4b0f-aea5-d2857b22bad3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:39.728837 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31925b34-f8f2-4b0f-aea5-d2857b22bad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:39.730840 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1126/2256] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:39 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:39.745687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bc370-b013-43c7-b08b-f8487d6e6382 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:39.747143 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f60bc370-b013-43c7-b08b-f8487d6e6382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:39.778027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-31925b34-f8f2-4b0f-aea5-d2857b22bad3 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:39.779273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-31925b34-f8f2-4b0f-aea5-d2857b22bad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:39.840846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:39.902657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bc370-b013-43c7-b08b-f8487d6e6382 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:39.923684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:39.952255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1131/2257] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:39.974670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31925b34-f8f2-4b0f-aea5-d2857b22bad3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:39.982142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-31925b34-f8f2-4b0f-aea5-d2857b22bad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:40.093397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f433d-3387-46db-aa26-34a60209a67c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:40.094021 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f433d-3387-46db-aa26-34a60209a67c service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:40.094312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f433d-3387-46db-aa26-34a60209a67c service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:40.094676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f433d-3387-46db-aa26-34a60209a67c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:40.099635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1e9f433d-3387-46db-aa26-34a60209a67c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:40.131165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31925b34-f8f2-4b0f-aea5-d2857b22bad3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:40.136814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425aac2-6d5f-445a-bf0e-d280daeeaa2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:40.137501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425aac2-6d5f-445a-bf0e-d280daeeaa2e service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:40.137933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425aac2-6d5f-445a-bf0e-d280daeeaa2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:40.138232 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425aac2-6d5f-445a-bf0e-d280daeeaa2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:40.160805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425aac2-6d5f-445a-bf0e-d280daeeaa2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:40.162331 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425aac2-6d5f-445a-bf0e-d280daeeaa2e service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:40.213361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1132/2258] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:11:40.250963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f433d-3387-46db-aa26-34a60209a67c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:40.252884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f433d-3387-46db-aa26-34a60209a67c service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:40.267263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1127/2259] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:11:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:11:40.709186 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425aac2-6d5f-445a-bf0e-d280daeeaa2e service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:40.711632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c425aac2-6d5f-445a-bf0e-d280daeeaa2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:40.741119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425aac2-6d5f-445a-bf0e-d280daeeaa2e service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:40.750382 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff57109-5a30-445b-95ee-37887ebfaa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:40.751595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff57109-5a30-445b-95ee-37887ebfaa92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:40.752139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff57109-5a30-445b-95ee-37887ebfaa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:40.753342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff57109-5a30-445b-95ee-37887ebfaa92 admin admin] PATH_INFO: `/v3/projects/3d0dea7ee2c840cb8ff7b4bd243c6da1` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:40.755293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff57109-5a30-445b-95ee-37887ebfaa92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d0dea7ee2c840cb8ff7b4bd243c6da1)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:40.757461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff57109-5a30-445b-95ee-37887ebfaa92 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:40.844267 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4ff57109-5a30-445b-95ee-37887ebfaa92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d0dea7ee2c840cb8ff7b4bd243c6da1', 'request_id': 'req-4ff57109-5a30-445b-95ee-37887ebfaa92'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:40.871168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4ff57109-5a30-445b-95ee-37887ebfaa92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d0dea7ee2c840cb8ff7b4bd243c6da1', 'request_id': 'req-4ff57109-5a30-445b-95ee-37887ebfaa92'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:40.878835 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4ff57109-5a30-445b-95ee-37887ebfaa92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d0dea7ee2c840cb8ff7b4bd243c6da1', 'request_id': 'req-4ff57109-5a30-445b-95ee-37887ebfaa92'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:40.908346 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4ff57109-5a30-445b-95ee-37887ebfaa92 admin admin] The token cache is being invalidate because project 3d0dea7ee2c840cb8ff7b4bd243c6da1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:11:40.908755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4ff57109-5a30-445b-95ee-37887ebfaa92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:40.909844 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4ff57109-5a30-445b-95ee-37887ebfaa92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:11:40.911767 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1133/2260] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:11:40 2022] DELETE /identity/v3/projects/3d0dea7ee2c840cb8ff7b4bd243c6da1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:11:41.781236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f71cd0-acb7-4b2c-bf68-9d09e9539f4d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:41.781874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f71cd0-acb7-4b2c-bf68-9d09e9539f4d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:41.782003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f71cd0-acb7-4b2c-bf68-9d09e9539f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:41.802738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c7f71cd0-acb7-4b2c-bf68-9d09e9539f4d None None] MFA Rules not processed for user `f7c009e1bcd74151ad3fa95c68916329`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:41.804114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c7f71cd0-acb7-4b2c-bf68-9d09e9539f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:41.867528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1128/2261] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:50.891260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:51.860088 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=87 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:11:58.181030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2110f2c-3487-4782-8e60-16a317a26764 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:58.181494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2110f2c-3487-4782-8e60-16a317a26764 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:58.181607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2110f2c-3487-4782-8e60-16a317a26764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:58.230703 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a2110f2c-3487-4782-8e60-16a317a26764 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:11:58.231905 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a2110f2c-3487-4782-8e60-16a317a26764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:58.297452 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1134/2262] 10.222.0.79 () {58 vars in 967 bytes} [Mon Jul 4 10:11:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:11:58.310330 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f71cd0-acb7-4b2c-bf68-9d09e9539f4d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:11:58.311951 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c7f71cd0-acb7-4b2c-bf68-9d09e9539f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:11:58.409432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f71cd0-acb7-4b2c-bf68-9d09e9539f4d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:11:58.415163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040e34b-5efd-4af1-97fe-bdcd22e76543 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:11:58.415816 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040e34b-5efd-4af1-97fe-bdcd22e76543 service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:11:58.416111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040e34b-5efd-4af1-97fe-bdcd22e76543 service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:11:58.416382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040e34b-5efd-4af1-97fe-bdcd22e76543 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:11:58.425180 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040e34b-5efd-4af1-97fe-bdcd22e76543 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:11:58.426819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040e34b-5efd-4af1-97fe-bdcd22e76543 service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:11:58.435066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1129/2263] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:11:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 10:12:08.294377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=40 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:12:08.431687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=41 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:12:17.434996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2110f2c-3487-4782-8e60-16a317a26764 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:12:17.436634 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a2110f2c-3487-4782-8e60-16a317a26764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:12:17.497643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2110f2c-3487-4782-8e60-16a317a26764 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:12:17.501515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95acca-7205-4c10-a6a7-20228b0ddf9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:12:17.502258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95acca-7205-4c10-a6a7-20228b0ddf9e service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:12:17.502535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95acca-7205-4c10-a6a7-20228b0ddf9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:12:17.502816 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95acca-7205-4c10-a6a7-20228b0ddf9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:12:17.504700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6a95acca-7205-4c10-a6a7-20228b0ddf9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:12:17.574968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95acca-7205-4c10-a6a7-20228b0ddf9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:12:17.576416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95acca-7205-4c10-a6a7-20228b0ddf9e service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:12:17.586343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1135/2264] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:12:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:12:25.158485 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf251c-d236-48e1-8c38-ba654a445a0c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:12:25.159003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf251c-d236-48e1-8c38-ba654a445a0c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:12:25.159115 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf251c-d236-48e1-8c38-ba654a445a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:12:25.177996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-bbcf251c-d236-48e1-8c38-ba654a445a0c None None] MFA Rules not processed for user `88d2351fc3b74ef5a530fded1be6b997`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:12:25.179190 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bbcf251c-d236-48e1-8c38-ba654a445a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:12:25.243991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1130/2265] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:12:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:12:27.581862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=50 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:12:35.237093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=25 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:12:45.298325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf29ec-9fdb-4992-a60f-d8395020dcec None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:12:45.298887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf29ec-9fdb-4992-a60f-d8395020dcec None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:12:45.298887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf29ec-9fdb-4992-a60f-d8395020dcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:12:45.317588 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-42cf29ec-9fdb-4992-a60f-d8395020dcec None None] MFA Rules not processed for user `fc9c26318d6c41f6b9455153f0d0e55b`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:12:45.318835 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-42cf29ec-9fdb-4992-a60f-d8395020dcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:12:45.380209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1136/2266] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:12:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:12:52.866114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1803aa-5ef5-4863-a530-ab4854a844ac None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:12:52.866673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1803aa-5ef5-4863-a530-ab4854a844ac None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:12:52.866673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1803aa-5ef5-4863-a530-ab4854a844ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:12:52.885300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1e1803aa-5ef5-4863-a530-ab4854a844ac None None] MFA Rules not processed for user `1c05b9b08df0437f9b0abb167d17aa56`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:12:52.886775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1e1803aa-5ef5-4863-a530-ab4854a844ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:12:52.949522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1131/2267] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:12:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:12:55.372828 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=25 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:13:02.942515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=25 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:13:24.427197 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b32a07-720b-433a-b683-a4ca2cb4c85c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:24.427881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b32a07-720b-433a-b683-a4ca2cb4c85c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:24.427881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b32a07-720b-433a-b683-a4ca2cb4c85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:24.445964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-68b32a07-720b-433a-b683-a4ca2cb4c85c None None] MFA Rules not processed for user `dfba262d439a4253bf1c2c0f114d0cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:13:24.447172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-68b32a07-720b-433a-b683-a4ca2cb4c85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:24.505977 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1137/2268] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:13:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:13:26.298939 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1803aa-5ef5-4863-a530-ab4854a844ac None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:26.300648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1e1803aa-5ef5-4863-a530-ab4854a844ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:26.316718 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1803aa-5ef5-4863-a530-ab4854a844ac None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:26.320675 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c9dc0-3f8a-4969-8f7b-faf83f5a2468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:26.321503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c9dc0-3f8a-4969-8f7b-faf83f5a2468 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:26.321897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c9dc0-3f8a-4969-8f7b-faf83f5a2468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:26.322238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c9dc0-3f8a-4969-8f7b-faf83f5a2468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:26.324103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7b6c9dc0-3f8a-4969-8f7b-faf83f5a2468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:26.405481 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c9dc0-3f8a-4969-8f7b-faf83f5a2468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:26.406908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c9dc0-3f8a-4969-8f7b-faf83f5a2468 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:26.419995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1132/2269] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:13:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:13:27.566519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b32a07-720b-433a-b683-a4ca2cb4c85c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:27.578687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b32a07-720b-433a-b683-a4ca2cb4c85c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:27.582597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6157e49-f9ea-4966-8063-a4cf34942f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:27.583258 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6157e49-f9ea-4966-8063-a4cf34942f49 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:27.583519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6157e49-f9ea-4966-8063-a4cf34942f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:27.583864 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6157e49-f9ea-4966-8063-a4cf34942f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:27.585949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a6157e49-f9ea-4966-8063-a4cf34942f49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:27.604270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6157e49-f9ea-4966-8063-a4cf34942f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:27.606028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6157e49-f9ea-4966-8063-a4cf34942f49 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:27.618374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1138/2270] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:13:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:13:36.420404 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=34 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:13:37.612867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=31 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:13:40.552046 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c9dc0-3f8a-4969-8f7b-faf83f5a2468 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:40.553601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7b6c9dc0-3f8a-4969-8f7b-faf83f5a2468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:40.567919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c9dc0-3f8a-4969-8f7b-faf83f5a2468 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:40.576056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94eec2-627c-44ff-b2db-acc8b263bb99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:40.576754 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94eec2-627c-44ff-b2db-acc8b263bb99 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:40.577005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94eec2-627c-44ff-b2db-acc8b263bb99 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:40.577284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94eec2-627c-44ff-b2db-acc8b263bb99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:40.579530 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5f94eec2-627c-44ff-b2db-acc8b263bb99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:40.592601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94eec2-627c-44ff-b2db-acc8b263bb99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:40.594473 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94eec2-627c-44ff-b2db-acc8b263bb99 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:40.607197 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1133/2271] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:13:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:13:45.054446 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25c9b0-ae2f-4c3e-a280-e7c64fd96fb7 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:45.055008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25c9b0-ae2f-4c3e-a280-e7c64fd96fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:45.055008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25c9b0-ae2f-4c3e-a280-e7c64fd96fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:45.074813 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-fa25c9b0-ae2f-4c3e-a280-e7c64fd96fb7 None None] MFA Rules not processed for user `88d2351fc3b74ef5a530fded1be6b997`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:13:45.076218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fa25c9b0-ae2f-4c3e-a280-e7c64fd96fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:45.085155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1139/2272] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:13:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:13:45.427111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94eec2-627c-44ff-b2db-acc8b263bb99 service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:45.437143 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94eec2-627c-44ff-b2db-acc8b263bb99 service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:45.441472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b5ae1-9919-4857-8cec-fa6e641280d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:45.442151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b5ae1-9919-4857-8cec-fa6e641280d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:45.442440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b5ae1-9919-4857-8cec-fa6e641280d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:45.442744 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b5ae1-9919-4857-8cec-fa6e641280d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:45.444871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-916b5ae1-9919-4857-8cec-fa6e641280d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:45.455553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b5ae1-9919-4857-8cec-fa6e641280d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:45.457067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b5ae1-9919-4857-8cec-fa6e641280d3 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:45.468835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1134/2273] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:13:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:13:46.749844 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25c9b0-ae2f-4c3e-a280-e7c64fd96fb7 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:46.762862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25c9b0-ae2f-4c3e-a280-e7c64fd96fb7 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:46.772720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd6253-b647-49f0-9581-7482efa3b25c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:46.773732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd6253-b647-49f0-9581-7482efa3b25c service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:46.775490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd6253-b647-49f0-9581-7482efa3b25c service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:46.776045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd6253-b647-49f0-9581-7482efa3b25c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:46.778449 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6abd6253-b647-49f0-9581-7482efa3b25c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:46.797101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd6253-b647-49f0-9581-7482efa3b25c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:46.800094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd6253-b647-49f0-9581-7482efa3b25c service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:46.816600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1140/2274] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:13:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:13:48.322094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b5ae1-9919-4857-8cec-fa6e641280d3 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:48.341979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b5ae1-9919-4857-8cec-fa6e641280d3 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:48.348723 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198e7f6-5789-4241-94fb-fd65e62f3797 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:48.349392 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e7f6-5789-4241-94fb-fd65e62f3797 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:48.349759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e7f6-5789-4241-94fb-fd65e62f3797 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:48.350061 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e7f6-5789-4241-94fb-fd65e62f3797 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:48.352018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6198e7f6-5789-4241-94fb-fd65e62f3797 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:48.363232 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198e7f6-5789-4241-94fb-fd65e62f3797 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:48.364876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198e7f6-5789-4241-94fb-fd65e62f3797 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:48.375052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1135/2275] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:13:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:13:52.684085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd6253-b647-49f0-9581-7482efa3b25c service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:52.693132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd6253-b647-49f0-9581-7482efa3b25c service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:52.697347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3458a-e5fc-41ad-8ab8-b292d910d9b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:52.698120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3458a-e5fc-41ad-8ab8-b292d910d9b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:52.698322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3458a-e5fc-41ad-8ab8-b292d910d9b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:52.698627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3458a-e5fc-41ad-8ab8-b292d910d9b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:52.700560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b8e3458a-e5fc-41ad-8ab8-b292d910d9b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:52.770894 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3458a-e5fc-41ad-8ab8-b292d910d9b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:52.772364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3458a-e5fc-41ad-8ab8-b292d910d9b4 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:52.783355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1141/2276] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:13:56.021120 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198e7f6-5789-4241-94fb-fd65e62f3797 service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:56.022821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6198e7f6-5789-4241-94fb-fd65e62f3797 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:56.101169 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3458a-e5fc-41ad-8ab8-b292d910d9b4 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:56.102669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b8e3458a-e5fc-41ad-8ab8-b292d910d9b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:56.106507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198e7f6-5789-4241-94fb-fd65e62f3797 service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:56.117158 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365a9d9-8e99-49e8-97d2-91c9d9359e53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:56.117421 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3458a-e5fc-41ad-8ab8-b292d910d9b4 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:56.117867 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365a9d9-8e99-49e8-97d2-91c9d9359e53 service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:56.118159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365a9d9-8e99-49e8-97d2-91c9d9359e53 service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:56.118435 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365a9d9-8e99-49e8-97d2-91c9d9359e53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:56.120822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f365a9d9-8e99-49e8-97d2-91c9d9359e53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:56.122717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2ec4c-aceb-4d5c-b86c-2e1b9d9ce1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:56.123601 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2ec4c-aceb-4d5c-b86c-2e1b9d9ce1ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:56.124041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2ec4c-aceb-4d5c-b86c-2e1b9d9ce1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:56.124484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2ec4c-aceb-4d5c-b86c-2e1b9d9ce1ab admin admin] PATH_INFO: `/v3/users/fc9c26318d6c41f6b9455153f0d0e55b` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:56.126007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2ec4c-aceb-4d5c-b86c-2e1b9d9ce1ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc9c26318d6c41f6b9455153f0d0e55b)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:56.128147 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2ec4c-aceb-4d5c-b86c-2e1b9d9ce1ab admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:56.139533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365a9d9-8e99-49e8-97d2-91c9d9359e53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:56.141213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365a9d9-8e99-49e8-97d2-91c9d9359e53 service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:56.152419 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1136/2277] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:13:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:13:56.182874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-72d2ec4c-aceb-4d5c-b86c-2e1b9d9ce1ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc9c26318d6c41f6b9455153f0d0e55b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:13:56.183466 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-72d2ec4c-aceb-4d5c-b86c-2e1b9d9ce1ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc9c26318d6c41f6b9455153f0d0e55b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:13:56.192456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-72d2ec4c-aceb-4d5c-b86c-2e1b9d9ce1ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc9c26318d6c41f6b9455153f0d0e55b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:13:56.206503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-72d2ec4c-aceb-4d5c-b86c-2e1b9d9ce1ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc9c26318d6c41f6b9455153f0d0e55b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:13:56.207219 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-72d2ec4c-aceb-4d5c-b86c-2e1b9d9ce1ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc9c26318d6c41f6b9455153f0d0e55b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:13:56.220449 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1142/2278] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:13:56 2022] DELETE /identity/v3/users/fc9c26318d6c41f6b9455153f0d0e55b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:13:56.417027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365a9d9-8e99-49e8-97d2-91c9d9359e53 service placement] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:56.429452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365a9d9-8e99-49e8-97d2-91c9d9359e53 service placement] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:56.489307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06556abf-21e4-48c0-9cc9-250296d8a3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:56.490220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06556abf-21e4-48c0-9cc9-250296d8a3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:56.491084 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06556abf-21e4-48c0-9cc9-250296d8a3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:56.491084 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06556abf-21e4-48c0-9cc9-250296d8a3d3 admin admin] PATH_INFO: `/v3/projects/7f101e48ae9b4936a6d1b7bae000d7e1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:56.492058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06556abf-21e4-48c0-9cc9-250296d8a3d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f101e48ae9b4936a6d1b7bae000d7e1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:56.494197 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06556abf-21e4-48c0-9cc9-250296d8a3d3 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:56.544461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-06556abf-21e4-48c0-9cc9-250296d8a3d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f101e48ae9b4936a6d1b7bae000d7e1', 'request_id': 'req-06556abf-21e4-48c0-9cc9-250296d8a3d3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:13:56.547224 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:13:56.552448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-06556abf-21e4-48c0-9cc9-250296d8a3d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f101e48ae9b4936a6d1b7bae000d7e1', 'request_id': 'req-06556abf-21e4-48c0-9cc9-250296d8a3d3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:13:56.559747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-06556abf-21e4-48c0-9cc9-250296d8a3d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f101e48ae9b4936a6d1b7bae000d7e1', 'request_id': 'req-06556abf-21e4-48c0-9cc9-250296d8a3d3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:13:56.577055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-06556abf-21e4-48c0-9cc9-250296d8a3d3 admin admin] The token cache is being invalidate because project 7f101e48ae9b4936a6d1b7bae000d7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:13:56.577353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-06556abf-21e4-48c0-9cc9-250296d8a3d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:13:56.578047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-06556abf-21e4-48c0-9cc9-250296d8a3d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:13:56.579738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1137/2279] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:13:56 2022] DELETE /identity/v3/projects/7f101e48ae9b4936a6d1b7bae000d7e1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:13:56.584776 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc65ab-cff2-422f-ba55-96b9529b9ae1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:56.585072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc65ab-cff2-422f-ba55-96b9529b9ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:56.585354 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc65ab-cff2-422f-ba55-96b9529b9ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:56.606296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7ffc65ab-cff2-422f-ba55-96b9529b9ae1 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:13:56.607863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7ffc65ab-cff2-422f-ba55-96b9529b9ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:56.662934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:13:56.687509 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1143/2280] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:13:56.695231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca56033-1a6e-4eab-b429-9587ff38cdd0 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:56.695647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca56033-1a6e-4eab-b429-9587ff38cdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:56.695997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca56033-1a6e-4eab-b429-9587ff38cdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:56.719220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9ca56033-1a6e-4eab-b429-9587ff38cdd0 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:13:56.720753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9ca56033-1a6e-4eab-b429-9587ff38cdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:56.734622 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1138/2281] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:13:56.735917 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc65ab-cff2-422f-ba55-96b9529b9ae1 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:56.749163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc65ab-cff2-422f-ba55-96b9529b9ae1 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:56.754009 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0b42f-de32-46ba-b059-32211ce5d3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:56.754680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0b42f-de32-46ba-b059-32211ce5d3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:56.754959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0b42f-de32-46ba-b059-32211ce5d3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:56.755228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0b42f-de32-46ba-b059-32211ce5d3d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:56.755991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0b42f-de32-46ba-b059-32211ce5d3d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:13:56.756295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0b42f-de32-46ba-b059-32211ce5d3d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:56.757924 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0b42f-de32-46ba-b059-32211ce5d3d8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:56.772760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1144/2282] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:13:56.777134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca56033-1a6e-4eab-b429-9587ff38cdd0 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:56.787000 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca56033-1a6e-4eab-b429-9587ff38cdd0 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:56.792492 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9b4aa-2683-4ee3-beb1-ec8001044e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:56.793153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9b4aa-2683-4ee3-beb1-ec8001044e51 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:56.793430 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9b4aa-2683-4ee3-beb1-ec8001044e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:56.793734 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9b4aa-2683-4ee3-beb1-ec8001044e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:56.794453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9b4aa-2683-4ee3-beb1-ec8001044e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:56.796806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9b4aa-2683-4ee3-beb1-ec8001044e51 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:56.828305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1139/2283] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:13:56 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:13:56.832541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0b42f-de32-46ba-b059-32211ce5d3d8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:56.843293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0b42f-de32-46ba-b059-32211ce5d3d8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:56.901799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0726a9-0925-49c0-8ef4-8d64d7e55efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:56.902458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0726a9-0925-49c0-8ef4-8d64d7e55efc admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:56.902744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0726a9-0925-49c0-8ef4-8d64d7e55efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:56.903019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0726a9-0925-49c0-8ef4-8d64d7e55efc admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:56.903705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0726a9-0925-49c0-8ef4-8d64d7e55efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:56.905660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0726a9-0925-49c0-8ef4-8d64d7e55efc admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:56.928010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1145/2284] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:13:56 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:13:56.932110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9b4aa-2683-4ee3-beb1-ec8001044e51 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:56.944603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9b4aa-2683-4ee3-beb1-ec8001044e51 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:56.950972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038da95e-b8d0-4ab9-b5cc-c33800ace092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:56.951628 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038da95e-b8d0-4ab9-b5cc-c33800ace092 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:56.951911 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038da95e-b8d0-4ab9-b5cc-c33800ace092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:56.952184 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038da95e-b8d0-4ab9-b5cc-c33800ace092 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:56.952921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038da95e-b8d0-4ab9-b5cc-c33800ace092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:56.954577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038da95e-b8d0-4ab9-b5cc-c33800ace092 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:56.966231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1140/2285] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:13:56.970548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0726a9-0925-49c0-8ef4-8d64d7e55efc admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:56.982320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0726a9-0925-49c0-8ef4-8d64d7e55efc admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:56.988451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f6fce-ec71-4e95-8ba1-ba811d02d390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:56.989114 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f6fce-ec71-4e95-8ba1-ba811d02d390 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:56.989394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f6fce-ec71-4e95-8ba1-ba811d02d390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:56.989807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f6fce-ec71-4e95-8ba1-ba811d02d390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:56.990529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f6fce-ec71-4e95-8ba1-ba811d02d390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:56.992178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f6fce-ec71-4e95-8ba1-ba811d02d390 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:57.003053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1146/2286] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:13:57.007610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038da95e-b8d0-4ab9-b5cc-c33800ace092 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:57.018355 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038da95e-b8d0-4ab9-b5cc-c33800ace092 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:57.024311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e2c32-a22c-47d2-8e18-2ba215ee3514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:57.025198 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e2c32-a22c-47d2-8e18-2ba215ee3514 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:57.025582 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e2c32-a22c-47d2-8e18-2ba215ee3514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:57.025863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e2c32-a22c-47d2-8e18-2ba215ee3514 admin admin] PATH_INFO: `/v3/projects/2022360469d84a1b8b721abab92d6b11/users/81005bd3b16e4599afea530732c9ceb9/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:57.041513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e2c32-a22c-47d2-8e18-2ba215ee3514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2022360469d84a1b8b721abab92d6b11, user_id=81005bd3b16e4599afea530732c9ceb9, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:57.044313 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e2c32-a22c-47d2-8e18-2ba215ee3514 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:57.060454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1141/2287] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:13:57 2022] PUT /identity/v3/projects/2022360469d84a1b8b721abab92d6b11/users/81005bd3b16e4599afea530732c9ceb9/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:13:57.073716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861c230-1706-4cb0-a8a9-30f992ee0f01 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:57.073987 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861c230-1706-4cb0-a8a9-30f992ee0f01 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:57.074294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861c230-1706-4cb0-a8a9-30f992ee0f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:57.102425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-9861c230-1706-4cb0-a8a9-30f992ee0f01 None None] MFA Rules not processed for user `81005bd3b16e4599afea530732c9ceb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:13:57.103663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9861c230-1706-4cb0-a8a9-30f992ee0f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:57.174780 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1147/2288] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:13:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:13:57.186432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e2c32-a22c-47d2-8e18-2ba215ee3514 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:13:57.188188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0b9e2c32-a22c-47d2-8e18-2ba215ee3514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:13:57.247570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e2c32-a22c-47d2-8e18-2ba215ee3514 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:13:57.251281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45373b2-f88e-4dc4-9ac6-d5a1271da167 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:13:57.251939 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45373b2-f88e-4dc4-9ac6-d5a1271da167 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:13:57.252222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45373b2-f88e-4dc4-9ac6-d5a1271da167 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:13:57.252498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45373b2-f88e-4dc4-9ac6-d5a1271da167 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:13:57.261299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45373b2-f88e-4dc4-9ac6-d5a1271da167 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:13:57.262880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45373b2-f88e-4dc4-9ac6-d5a1271da167 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:13:57.271720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1142/2289] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:13:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:00.104399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861c230-1706-4cb0-a8a9-30f992ee0f01 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:00.106004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9861c230-1706-4cb0-a8a9-30f992ee0f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:00.161526 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:00.174662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861c230-1706-4cb0-a8a9-30f992ee0f01 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:00.178479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71375600-7afd-4cac-a6a5-aeabba8a0d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:00.179136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71375600-7afd-4cac-a6a5-aeabba8a0d4c service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:00.179418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71375600-7afd-4cac-a6a5-aeabba8a0d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:00.179694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71375600-7afd-4cac-a6a5-aeabba8a0d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:00.181583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-71375600-7afd-4cac-a6a5-aeabba8a0d4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:00.192779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71375600-7afd-4cac-a6a5-aeabba8a0d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:00.194262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71375600-7afd-4cac-a6a5-aeabba8a0d4c service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:00.202859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1148/2290] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:04.451418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd9d3e-1be2-427c-b657-87110906c661 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:04.451877 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd9d3e-1be2-427c-b657-87110906c661 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:04.451984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd9d3e-1be2-427c-b657-87110906c661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:04.471178 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2bcd9d3e-1be2-427c-b657-87110906c661 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:04.472437 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2bcd9d3e-1be2-427c-b657-87110906c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:04.531587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1143/2291] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:14:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:04.586571 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71375600-7afd-4cac-a6a5-aeabba8a0d4c service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:04.588116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-71375600-7afd-4cac-a6a5-aeabba8a0d4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:04.680808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71375600-7afd-4cac-a6a5-aeabba8a0d4c service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:04.685351 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e2013-c728-450d-9e1e-65a254af83c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:04.686057 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e2013-c728-450d-9e1e-65a254af83c5 service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:04.686341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e2013-c728-450d-9e1e-65a254af83c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:04.686622 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e2013-c728-450d-9e1e-65a254af83c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:04.695614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e2013-c728-450d-9e1e-65a254af83c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:04.697265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e2013-c728-450d-9e1e-65a254af83c5 service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:04.706520 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1149/2292] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:14:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 10:14:05.707755 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd9d3e-1be2-427c-b657-87110906c661 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:05.740469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:05.787605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd9d3e-1be2-427c-b657-87110906c661 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:05.795457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:05.796362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:05.796651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:05.796932 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] PATH_INFO: `/v3/users/81005bd3b16e4599afea530732c9ceb9` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:05.799624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81005bd3b16e4599afea530732c9ceb9)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:05.801756 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:05.884853 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81005bd3b16e4599afea530732c9ceb9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:05.885499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81005bd3b16e4599afea530732c9ceb9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:05.896869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81005bd3b16e4599afea530732c9ceb9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:05.912952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81005bd3b16e4599afea530732c9ceb9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:05.914024 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81005bd3b16e4599afea530732c9ceb9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:05.928604 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1144/2293] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:14:05 2022] DELETE /identity/v3/users/81005bd3b16e4599afea530732c9ceb9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:05.934693 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2813ab2-d631-4df2-b2b5-cff27c3d0eff None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:05.935298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2813ab2-d631-4df2-b2b5-cff27c3d0eff None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:05.935548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2813ab2-d631-4df2-b2b5-cff27c3d0eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:05.958116 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f2813ab2-d631-4df2-b2b5-cff27c3d0eff None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:05.959619 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f2813ab2-d631-4df2-b2b5-cff27c3d0eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:06.031408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1150/2294] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:14:06.041799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:06.043564 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:06.113340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b40cc1-e9bb-4c9d-b6e5-8f7862c369d1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:06.117451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0d064-2735-4ac2-bb43-7f7dd770b909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:06.118319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0d064-2735-4ac2-bb43-7f7dd770b909 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:06.118601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0d064-2735-4ac2-bb43-7f7dd770b909 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:06.118878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0d064-2735-4ac2-bb43-7f7dd770b909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:06.137707 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0d064-2735-4ac2-bb43-7f7dd770b909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:06.139279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0d064-2735-4ac2-bb43-7f7dd770b909 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:06.151076 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1145/2295] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:06.480956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2813ab2-d631-4df2-b2b5-cff27c3d0eff None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:06.482752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f2813ab2-d631-4df2-b2b5-cff27c3d0eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:06.496143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2813ab2-d631-4df2-b2b5-cff27c3d0eff None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:06.518669 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5211daa-08d0-4ea9-865e-1a8c38e956ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:06.519603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5211daa-08d0-4ea9-865e-1a8c38e956ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:06.519834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5211daa-08d0-4ea9-865e-1a8c38e956ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:06.520111 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5211daa-08d0-4ea9-865e-1a8c38e956ac admin admin] PATH_INFO: `/v3/projects/2022360469d84a1b8b721abab92d6b11` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:06.521347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5211daa-08d0-4ea9-865e-1a8c38e956ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2022360469d84a1b8b721abab92d6b11)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:06.523490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5211daa-08d0-4ea9-865e-1a8c38e956ac admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:06.591229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e5211daa-08d0-4ea9-865e-1a8c38e956ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2022360469d84a1b8b721abab92d6b11', 'request_id': 'req-e5211daa-08d0-4ea9-865e-1a8c38e956ac'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:06.597489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e5211daa-08d0-4ea9-865e-1a8c38e956ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2022360469d84a1b8b721abab92d6b11', 'request_id': 'req-e5211daa-08d0-4ea9-865e-1a8c38e956ac'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:06.608618 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e5211daa-08d0-4ea9-865e-1a8c38e956ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2022360469d84a1b8b721abab92d6b11', 'request_id': 'req-e5211daa-08d0-4ea9-865e-1a8c38e956ac'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:06.626303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:06.633053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e5211daa-08d0-4ea9-865e-1a8c38e956ac admin admin] The token cache is being invalidate because project 2022360469d84a1b8b721abab92d6b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:14:06.633053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e5211daa-08d0-4ea9-865e-1a8c38e956ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:06.634538 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e5211daa-08d0-4ea9-865e-1a8c38e956ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:06.636625 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1151/2296] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:14:06 2022] DELETE /identity/v3/projects/2022360469d84a1b8b721abab92d6b11 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:06.641587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383af21c-e37c-46f7-96e5-471a4a19c8ed None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:06.641901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383af21c-e37c-46f7-96e5-471a4a19c8ed None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:06.642197 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383af21c-e37c-46f7-96e5-471a4a19c8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:06.665471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-383af21c-e37c-46f7-96e5-471a4a19c8ed None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:06.666803 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-383af21c-e37c-46f7-96e5-471a4a19c8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:06.721656 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:06.749730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1146/2297] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:06.756072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb9091-9ff8-45c1-9f2a-428e59864cf9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:06.756362 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb9091-9ff8-45c1-9f2a-428e59864cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:06.756641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb9091-9ff8-45c1-9f2a-428e59864cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:06.781163 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b2eb9091-9ff8-45c1-9f2a-428e59864cf9 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:06.782587 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b2eb9091-9ff8-45c1-9f2a-428e59864cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:06.791537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1152/2298] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:06.796094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383af21c-e37c-46f7-96e5-471a4a19c8ed None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:06.807832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383af21c-e37c-46f7-96e5-471a4a19c8ed None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:06.813228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e84fd-8704-4263-ad79-3a197af3c5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:06.813981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e84fd-8704-4263-ad79-3a197af3c5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:06.814273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e84fd-8704-4263-ad79-3a197af3c5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:06.814552 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e84fd-8704-4263-ad79-3a197af3c5f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:06.815357 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e84fd-8704-4263-ad79-3a197af3c5f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:14:06.815623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e84fd-8704-4263-ad79-3a197af3c5f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:06.817227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e84fd-8704-4263-ad79-3a197af3c5f5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:06.833664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1147/2299] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 10:14:06.836745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb9091-9ff8-45c1-9f2a-428e59864cf9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:06.847043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb9091-9ff8-45c1-9f2a-428e59864cf9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:06.852312 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f139d09-face-4e9a-92ea-9d0e24e3e94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:06.853269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f139d09-face-4e9a-92ea-9d0e24e3e94e admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:06.853645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f139d09-face-4e9a-92ea-9d0e24e3e94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:06.854008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f139d09-face-4e9a-92ea-9d0e24e3e94e admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:06.855582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f139d09-face-4e9a-92ea-9d0e24e3e94e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:06.857461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f139d09-face-4e9a-92ea-9d0e24e3e94e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:06.889279 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1153/2300] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:14:06 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:14:06.893951 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e84fd-8704-4263-ad79-3a197af3c5f5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:06.907777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e84fd-8704-4263-ad79-3a197af3c5f5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:06.972933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e65a6-6933-4bb1-ab2e-1ad75eb515d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:06.973662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e65a6-6933-4bb1-ab2e-1ad75eb515d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:06.973952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e65a6-6933-4bb1-ab2e-1ad75eb515d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:06.974234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e65a6-6933-4bb1-ab2e-1ad75eb515d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:06.974941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e65a6-6933-4bb1-ab2e-1ad75eb515d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:06.978817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e65a6-6933-4bb1-ab2e-1ad75eb515d2 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:07.004544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1148/2301] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:14:06 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:14:07.008728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f139d09-face-4e9a-92ea-9d0e24e3e94e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:07.020821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f139d09-face-4e9a-92ea-9d0e24e3e94e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:07.025747 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1d08b-c0bf-4e74-8772-f4799907de86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:07.026769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1d08b-c0bf-4e74-8772-f4799907de86 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:07.027157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1d08b-c0bf-4e74-8772-f4799907de86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:07.028132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1d08b-c0bf-4e74-8772-f4799907de86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:07.029004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1d08b-c0bf-4e74-8772-f4799907de86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:07.030804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1d08b-c0bf-4e74-8772-f4799907de86 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:07.046058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1154/2302] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:14:07.046852 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e65a6-6933-4bb1-ab2e-1ad75eb515d2 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:07.057997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e65a6-6933-4bb1-ab2e-1ad75eb515d2 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:07.062839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a525627-1fe8-4863-923a-360f0c82ce28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:07.063517 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525627-1fe8-4863-923a-360f0c82ce28 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:07.063818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525627-1fe8-4863-923a-360f0c82ce28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:07.064075 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525627-1fe8-4863-923a-360f0c82ce28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:07.065056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a525627-1fe8-4863-923a-360f0c82ce28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:07.066715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a525627-1fe8-4863-923a-360f0c82ce28 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:07.081654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1149/2303] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:14:07.084833 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1d08b-c0bf-4e74-8772-f4799907de86 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:07.096153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1d08b-c0bf-4e74-8772-f4799907de86 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:07.101128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449c6d6-1034-4d2c-866e-a1bf6b386815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:07.102069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449c6d6-1034-4d2c-866e-a1bf6b386815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:07.102357 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449c6d6-1034-4d2c-866e-a1bf6b386815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:07.102632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449c6d6-1034-4d2c-866e-a1bf6b386815 admin admin] PATH_INFO: `/v3/projects/b769792ac0634a4f863e0e594c6e55ac/users/3a71acc8a61b4021b5fe8a0ca29b1b84/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:07.120260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449c6d6-1034-4d2c-866e-a1bf6b386815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b769792ac0634a4f863e0e594c6e55ac, user_id=3a71acc8a61b4021b5fe8a0ca29b1b84, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:07.123036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449c6d6-1034-4d2c-866e-a1bf6b386815 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:07.147646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1155/2304] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:14:07 2022] PUT /identity/v3/projects/b769792ac0634a4f863e0e594c6e55ac/users/3a71acc8a61b4021b5fe8a0ca29b1b84/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:07.159389 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4754e-5f2f-4420-b357-9c2e8ee17c8e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:07.159698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4754e-5f2f-4420-b357-9c2e8ee17c8e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:07.159970 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4754e-5f2f-4420-b357-9c2e8ee17c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:07.192096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c2c4754e-5f2f-4420-b357-9c2e8ee17c8e None None] MFA Rules not processed for user `3a71acc8a61b4021b5fe8a0ca29b1b84`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:07.193356 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c2c4754e-5f2f-4420-b357-9c2e8ee17c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:07.276102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1150/2305] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:07.281160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449c6d6-1034-4d2c-866e-a1bf6b386815 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:07.359986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449c6d6-1034-4d2c-866e-a1bf6b386815 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:07.364738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f376723-41ab-4477-bc66-f12cdc65937b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:07.365406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f376723-41ab-4477-bc66-f12cdc65937b admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:07.365718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f376723-41ab-4477-bc66-f12cdc65937b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:07.365997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f376723-41ab-4477-bc66-f12cdc65937b admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:07.366685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f376723-41ab-4477-bc66-f12cdc65937b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:07.368608 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f376723-41ab-4477-bc66-f12cdc65937b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:07.396054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1156/2306] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:14:07 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:14:07.400475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4754e-5f2f-4420-b357-9c2e8ee17c8e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:07.413418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4754e-5f2f-4420-b357-9c2e8ee17c8e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:07.466160 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:07.491218 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fb7f8-fb66-40ba-bd5e-ee7da7d32968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:07.491923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fb7f8-fb66-40ba-bd5e-ee7da7d32968 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:07.492282 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fb7f8-fb66-40ba-bd5e-ee7da7d32968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:07.492618 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fb7f8-fb66-40ba-bd5e-ee7da7d32968 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:07.493783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fb7f8-fb66-40ba-bd5e-ee7da7d32968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:07.496365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fb7f8-fb66-40ba-bd5e-ee7da7d32968 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:07.522881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1151/2307] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:14:07 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:14:07.526815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f376723-41ab-4477-bc66-f12cdc65937b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:07.539850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f376723-41ab-4477-bc66-f12cdc65937b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:07.545058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db746dc0-b7b7-4b50-878b-bba7c0184b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:07.545771 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db746dc0-b7b7-4b50-878b-bba7c0184b6f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:07.546050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db746dc0-b7b7-4b50-878b-bba7c0184b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:07.546321 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db746dc0-b7b7-4b50-878b-bba7c0184b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:07.547058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db746dc0-b7b7-4b50-878b-bba7c0184b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:07.548679 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db746dc0-b7b7-4b50-878b-bba7c0184b6f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:07.565107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1157/2308] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:14:07.566449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fb7f8-fb66-40ba-bd5e-ee7da7d32968 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:07.577687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fb7f8-fb66-40ba-bd5e-ee7da7d32968 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:07.582410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f93ccb-0400-4766-a35d-2bad20439bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:07.583335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f93ccb-0400-4766-a35d-2bad20439bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:07.583603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f93ccb-0400-4766-a35d-2bad20439bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:07.583876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f93ccb-0400-4766-a35d-2bad20439bc9 admin admin] PATH_INFO: `/v3/projects/7f0846a1f5744644a69b2a171fd4b6df/users/3175bba4c7194da293f414a2d8adc7b2/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:07.601803 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f93ccb-0400-4766-a35d-2bad20439bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0846a1f5744644a69b2a171fd4b6df, user_id=3175bba4c7194da293f414a2d8adc7b2, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:07.605651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f93ccb-0400-4766-a35d-2bad20439bc9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:07.625303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1152/2309] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:14:07 2022] PUT /identity/v3/projects/7f0846a1f5744644a69b2a171fd4b6df/users/3175bba4c7194da293f414a2d8adc7b2/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:07.629444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db746dc0-b7b7-4b50-878b-bba7c0184b6f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:07.640807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db746dc0-b7b7-4b50-878b-bba7c0184b6f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:07.709168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcf66e-b26f-4d18-a4a7-51cbb58bcdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:07.710124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcf66e-b26f-4d18-a4a7-51cbb58bcdc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:07.710435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcf66e-b26f-4d18-a4a7-51cbb58bcdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:07.710687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcf66e-b26f-4d18-a4a7-51cbb58bcdc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:07.711424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcf66e-b26f-4d18-a4a7-51cbb58bcdc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:07.713241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcf66e-b26f-4d18-a4a7-51cbb58bcdc6 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:07.728977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f93ccb-0400-4766-a35d-2bad20439bc9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:07.729886 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1158/2310] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:14:07.739602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f93ccb-0400-4766-a35d-2bad20439bc9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:07.745791 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f662b-c66d-47a8-ab44-63c44adf9c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:07.747333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f662b-c66d-47a8-ab44-63c44adf9c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:07.747333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f662b-c66d-47a8-ab44-63c44adf9c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:07.747558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f662b-c66d-47a8-ab44-63c44adf9c7e admin admin] PATH_INFO: `/v3/projects/7f0846a1f5744644a69b2a171fd4b6df/users/3175bba4c7194da293f414a2d8adc7b2/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:07.749441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f662b-c66d-47a8-ab44-63c44adf9c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0846a1f5744644a69b2a171fd4b6df, user_id=3175bba4c7194da293f414a2d8adc7b2, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:07.752333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f662b-c66d-47a8-ab44-63c44adf9c7e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:07.774495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8ee0b-095c-4010-a714-1fe9bf6a849a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:07.775377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8ee0b-095c-4010-a714-1fe9bf6a849a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:07.775377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8ee0b-095c-4010-a714-1fe9bf6a849a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:07.777064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1153/2311] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:14:07 2022] PUT /identity/v3/projects/7f0846a1f5744644a69b2a171fd4b6df/users/3175bba4c7194da293f414a2d8adc7b2/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:07.782214 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:07.809732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-69e8ee0b-095c-4010-a714-1fe9bf6a849a None None] MFA Rules not processed for user `3175bba4c7194da293f414a2d8adc7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:07.810983 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-69e8ee0b-095c-4010-a714-1fe9bf6a849a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:07.911286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1159/2312] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:07 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:07.922769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f662b-c66d-47a8-ab44-63c44adf9c7e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:07.924621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2f0f662b-c66d-47a8-ab44-63c44adf9c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:07.999187 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f662b-c66d-47a8-ab44-63c44adf9c7e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:08.003427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926d72d-d654-42b7-95e0-8124e68e72c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:08.004087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926d72d-d654-42b7-95e0-8124e68e72c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:08.004370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926d72d-d654-42b7-95e0-8124e68e72c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:08.004644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926d72d-d654-42b7-95e0-8124e68e72c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:08.019867 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926d72d-d654-42b7-95e0-8124e68e72c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:08.021596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926d72d-d654-42b7-95e0-8124e68e72c8 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:08.036466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1154/2313] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:08.516083 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8ee0b-095c-4010-a714-1fe9bf6a849a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:08.597071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8ee0b-095c-4010-a714-1fe9bf6a849a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:08.602047 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:08.603146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:08.603609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:08.604062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] PATH_INFO: `/v3/users/3a71acc8a61b4021b5fe8a0ca29b1b84` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:08.605576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a71acc8a61b4021b5fe8a0ca29b1b84)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:08.607776 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:08.670062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a71acc8a61b4021b5fe8a0ca29b1b84'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:08.672125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a71acc8a61b4021b5fe8a0ca29b1b84'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:08.685224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a71acc8a61b4021b5fe8a0ca29b1b84'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:08.705581 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a71acc8a61b4021b5fe8a0ca29b1b84'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:08.706366 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a71acc8a61b4021b5fe8a0ca29b1b84'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:08.722917 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1160/2314] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:14:08 2022] DELETE /identity/v3/users/3a71acc8a61b4021b5fe8a0ca29b1b84 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:08.727268 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e13f50-18bf-4525-b658-65585a30b00b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:08.727637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e13f50-18bf-4525-b658-65585a30b00b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:08.727977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e13f50-18bf-4525-b658-65585a30b00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:08.747689 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e1e13f50-18bf-4525-b658-65585a30b00b None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:08.749080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e1e13f50-18bf-4525-b658-65585a30b00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:08.824793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1155/2315] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:08.837590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:08.839165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:08.851698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:08.933017 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963ce11-74e4-41b4-8d1f-cdbfefded753 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:08.938154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e1339-6cb6-4276-a333-a4d2ca4260fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:08.938875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e1339-6cb6-4276-a333-a4d2ca4260fc service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:08.939222 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e1339-6cb6-4276-a333-a4d2ca4260fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:08.939667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e1339-6cb6-4276-a333-a4d2ca4260fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:08.951923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e1339-6cb6-4276-a333-a4d2ca4260fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:08.953467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e1339-6cb6-4276-a333-a4d2ca4260fc service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:08.963707 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1161/2316] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:09.376926 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e13f50-18bf-4525-b658-65585a30b00b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:09.378680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e1e13f50-18bf-4525-b658-65585a30b00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:09.397507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e13f50-18bf-4525-b658-65585a30b00b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:09.411100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:09.412018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:09.412305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:09.412617 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] PATH_INFO: `/v3/projects/b769792ac0634a4f863e0e594c6e55ac` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:09.414030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b769792ac0634a4f863e0e594c6e55ac)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:09.415977 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:09.460085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:09.478938 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b769792ac0634a4f863e0e594c6e55ac', 'request_id': 'req-922dc920-5181-4e33-b250-52e8c97b4dd9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:09.489031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b769792ac0634a4f863e0e594c6e55ac', 'request_id': 'req-922dc920-5181-4e33-b250-52e8c97b4dd9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:09.495674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b769792ac0634a4f863e0e594c6e55ac', 'request_id': 'req-922dc920-5181-4e33-b250-52e8c97b4dd9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:09.513949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] The token cache is being invalidate because project b769792ac0634a4f863e0e594c6e55ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:14:09.514273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:09.514871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:09.516893 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1156/2317] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:14:09 2022] DELETE /identity/v3/projects/b769792ac0634a4f863e0e594c6e55ac => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:09.522698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e1339-6cb6-4276-a333-a4d2ca4260fc service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:09.534876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e1339-6cb6-4276-a333-a4d2ca4260fc service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:09.606213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd76dfa-1d68-4c13-b9a3-d91be0ef185f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:09.607280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd76dfa-1d68-4c13-b9a3-d91be0ef185f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:09.607659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd76dfa-1d68-4c13-b9a3-d91be0ef185f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:09.608000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd76dfa-1d68-4c13-b9a3-d91be0ef185f admin admin] PATH_INFO: `/v3/users/3175bba4c7194da293f414a2d8adc7b2` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:09.609267 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd76dfa-1d68-4c13-b9a3-d91be0ef185f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3175bba4c7194da293f414a2d8adc7b2)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:09.611871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd76dfa-1d68-4c13-b9a3-d91be0ef185f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:09.665853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4dd76dfa-1d68-4c13-b9a3-d91be0ef185f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3175bba4c7194da293f414a2d8adc7b2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:09.666620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4dd76dfa-1d68-4c13-b9a3-d91be0ef185f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3175bba4c7194da293f414a2d8adc7b2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:09.677635 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4dd76dfa-1d68-4c13-b9a3-d91be0ef185f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3175bba4c7194da293f414a2d8adc7b2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:09.697973 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4dd76dfa-1d68-4c13-b9a3-d91be0ef185f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3175bba4c7194da293f414a2d8adc7b2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:09.699150 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4dd76dfa-1d68-4c13-b9a3-d91be0ef185f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3175bba4c7194da293f414a2d8adc7b2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:09.718654 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1162/2318] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:14:09 2022] DELETE /identity/v3/users/3175bba4c7194da293f414a2d8adc7b2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:10.074780 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:10.076481 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:10.098145 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922dc920-5181-4e33-b250-52e8c97b4dd9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:10.170924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b48b9-8b2c-4e38-997c-b9d8da4b8842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:10.171813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b48b9-8b2c-4e38-997c-b9d8da4b8842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:10.172095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b48b9-8b2c-4e38-997c-b9d8da4b8842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:10.172371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b48b9-8b2c-4e38-997c-b9d8da4b8842 admin admin] PATH_INFO: `/v3/projects/7f0846a1f5744644a69b2a171fd4b6df` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:10.173563 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b48b9-8b2c-4e38-997c-b9d8da4b8842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f0846a1f5744644a69b2a171fd4b6df)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:10.176640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b48b9-8b2c-4e38-997c-b9d8da4b8842 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:10.246652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-909b48b9-8b2c-4e38-997c-b9d8da4b8842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f0846a1f5744644a69b2a171fd4b6df', 'request_id': 'req-909b48b9-8b2c-4e38-997c-b9d8da4b8842'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:10.254308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-909b48b9-8b2c-4e38-997c-b9d8da4b8842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f0846a1f5744644a69b2a171fd4b6df', 'request_id': 'req-909b48b9-8b2c-4e38-997c-b9d8da4b8842'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:10.261822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-909b48b9-8b2c-4e38-997c-b9d8da4b8842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f0846a1f5744644a69b2a171fd4b6df', 'request_id': 'req-909b48b9-8b2c-4e38-997c-b9d8da4b8842'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:10.290686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-909b48b9-8b2c-4e38-997c-b9d8da4b8842 admin admin] The token cache is being invalidate because project 7f0846a1f5744644a69b2a171fd4b6df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:14:10.291047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-909b48b9-8b2c-4e38-997c-b9d8da4b8842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:10.291769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-909b48b9-8b2c-4e38-997c-b9d8da4b8842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:10.294303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1157/2319] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:14:10 2022] DELETE /identity/v3/projects/7f0846a1f5744644a69b2a171fd4b6df => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:10.299801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5c71e-5bb7-4b02-bf4a-246044817488 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:10.300098 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5c71e-5bb7-4b02-bf4a-246044817488 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:10.300639 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5c71e-5bb7-4b02-bf4a-246044817488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:10.322169 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1df5c71e-5bb7-4b02-bf4a-246044817488 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:10.323543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1df5c71e-5bb7-4b02-bf4a-246044817488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:10.368057 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:10.436030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1163/2320] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:10.443578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9891d2b-c7a0-45f9-82a0-c364961a911b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:10.444013 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9891d2b-c7a0-45f9-82a0-c364961a911b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:10.444356 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9891d2b-c7a0-45f9-82a0-c364961a911b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:10.467205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b9891d2b-c7a0-45f9-82a0-c364961a911b None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:10.468446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b9891d2b-c7a0-45f9-82a0-c364961a911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:10.483932 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1158/2321] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:14:10.488069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5c71e-5bb7-4b02-bf4a-246044817488 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:10.499179 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5c71e-5bb7-4b02-bf4a-246044817488 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:10.503740 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5fe24-dda0-4aa8-8b68-b4322ada4784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:10.504412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5fe24-dda0-4aa8-8b68-b4322ada4784 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:10.504700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5fe24-dda0-4aa8-8b68-b4322ada4784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:10.504980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5fe24-dda0-4aa8-8b68-b4322ada4784 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:10.505738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5fe24-dda0-4aa8-8b68-b4322ada4784 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:14:10.506060 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5fe24-dda0-4aa8-8b68-b4322ada4784 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:10.507681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5fe24-dda0-4aa8-8b68-b4322ada4784 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:10.521063 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1164/2322] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:14:10.525486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9891d2b-c7a0-45f9-82a0-c364961a911b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:10.537392 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9891d2b-c7a0-45f9-82a0-c364961a911b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:10.542969 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef342cf-c709-4138-bb5c-78fcc5234396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:10.543769 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef342cf-c709-4138-bb5c-78fcc5234396 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:10.544243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef342cf-c709-4138-bb5c-78fcc5234396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:10.544692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef342cf-c709-4138-bb5c-78fcc5234396 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:10.545400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef342cf-c709-4138-bb5c-78fcc5234396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:10.547448 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef342cf-c709-4138-bb5c-78fcc5234396 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:10.581726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1159/2323] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:14:10 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:14:10.585689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5fe24-dda0-4aa8-8b68-b4322ada4784 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:10.597652 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5fe24-dda0-4aa8-8b68-b4322ada4784 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:10.663074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c3b3d-5740-4b1b-8587-5323d161a8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:10.663760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c3b3d-5740-4b1b-8587-5323d161a8ba admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:10.664047 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c3b3d-5740-4b1b-8587-5323d161a8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:10.664323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c3b3d-5740-4b1b-8587-5323d161a8ba admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:10.665034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c3b3d-5740-4b1b-8587-5323d161a8ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:10.667585 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c3b3d-5740-4b1b-8587-5323d161a8ba admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:10.699666 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1165/2324] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:14:10 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:14:10.702200 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef342cf-c709-4138-bb5c-78fcc5234396 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:10.713397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef342cf-c709-4138-bb5c-78fcc5234396 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:10.718863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279904bd-ac6b-4d80-a2a5-cebd7e8e175b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:10.719572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279904bd-ac6b-4d80-a2a5-cebd7e8e175b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:10.719854 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279904bd-ac6b-4d80-a2a5-cebd7e8e175b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:10.720131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279904bd-ac6b-4d80-a2a5-cebd7e8e175b admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:10.720869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279904bd-ac6b-4d80-a2a5-cebd7e8e175b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:10.723673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279904bd-ac6b-4d80-a2a5-cebd7e8e175b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:10.739272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1160/2325] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:14:10.740148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c3b3d-5740-4b1b-8587-5323d161a8ba admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:10.754338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c3b3d-5740-4b1b-8587-5323d161a8ba admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:10.759284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd837029-a4a9-4cbf-b16a-b72ad7fdcc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:10.759951 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd837029-a4a9-4cbf-b16a-b72ad7fdcc31 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:10.760239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd837029-a4a9-4cbf-b16a-b72ad7fdcc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:10.760522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd837029-a4a9-4cbf-b16a-b72ad7fdcc31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:10.761764 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd837029-a4a9-4cbf-b16a-b72ad7fdcc31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:10.763699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd837029-a4a9-4cbf-b16a-b72ad7fdcc31 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:10.779414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1166/2326] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:14:10.780508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279904bd-ac6b-4d80-a2a5-cebd7e8e175b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:10.796187 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279904bd-ac6b-4d80-a2a5-cebd7e8e175b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:10.804267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682eaf2-9069-4950-8742-e5a74cff1c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:10.805191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682eaf2-9069-4950-8742-e5a74cff1c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:10.805481 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682eaf2-9069-4950-8742-e5a74cff1c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:10.805804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682eaf2-9069-4950-8742-e5a74cff1c80 admin admin] PATH_INFO: `/v3/projects/78ac940025664893841da6ea5778863c/users/743d67814a3f4ac69bc7dbca50ad834b/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:10.818145 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:10.827034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682eaf2-9069-4950-8742-e5a74cff1c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ac940025664893841da6ea5778863c, user_id=743d67814a3f4ac69bc7dbca50ad834b, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:10.829943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682eaf2-9069-4950-8742-e5a74cff1c80 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:10.853383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1161/2327] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:14:10 2022] PUT /identity/v3/projects/78ac940025664893841da6ea5778863c/users/743d67814a3f4ac69bc7dbca50ad834b/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:14:10.866004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73013cb6-3bf3-42c1-a3e6-7da7f3c1096a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:10.866356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73013cb6-3bf3-42c1-a3e6-7da7f3c1096a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:10.866732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73013cb6-3bf3-42c1-a3e6-7da7f3c1096a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:10.899271 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-73013cb6-3bf3-42c1-a3e6-7da7f3c1096a None None] MFA Rules not processed for user `743d67814a3f4ac69bc7dbca50ad834b`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:10.900613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-73013cb6-3bf3-42c1-a3e6-7da7f3c1096a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:10.991277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682eaf2-9069-4950-8742-e5a74cff1c80 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:10.991789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1167/2328] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:11.068342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682eaf2-9069-4950-8742-e5a74cff1c80 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:11.073450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d125b7a-7594-4b70-ae16-ed6aa2088ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:11.074425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d125b7a-7594-4b70-ae16-ed6aa2088ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:11.074726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d125b7a-7594-4b70-ae16-ed6aa2088ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:11.075010 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d125b7a-7594-4b70-ae16-ed6aa2088ed1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:11.075729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d125b7a-7594-4b70-ae16-ed6aa2088ed1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:11.077755 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d125b7a-7594-4b70-ae16-ed6aa2088ed1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:11.109683 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1162/2329] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:14:10 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:14:11.111613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73013cb6-3bf3-42c1-a3e6-7da7f3c1096a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:11.123634 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73013cb6-3bf3-42c1-a3e6-7da7f3c1096a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:11.159446 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:11.198279 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f036d59-37df-46db-8326-b02748513460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:11.199583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f036d59-37df-46db-8326-b02748513460 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:11.199867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f036d59-37df-46db-8326-b02748513460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:11.200144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f036d59-37df-46db-8326-b02748513460 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:11.200844 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f036d59-37df-46db-8326-b02748513460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:11.202862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f036d59-37df-46db-8326-b02748513460 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:11.228736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1168/2330] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:14:11 2022] POST /identity/v3/users => generated 300 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:14:11.233818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d125b7a-7594-4b70-ae16-ed6aa2088ed1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:11.245939 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d125b7a-7594-4b70-ae16-ed6aa2088ed1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:11.251533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475861b-25eb-49f4-ae70-dfed7bb44a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:11.253012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475861b-25eb-49f4-ae70-dfed7bb44a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:11.253419 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475861b-25eb-49f4-ae70-dfed7bb44a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:11.253734 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475861b-25eb-49f4-ae70-dfed7bb44a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:11.254488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475861b-25eb-49f4-ae70-dfed7bb44a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:11.256417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475861b-25eb-49f4-ae70-dfed7bb44a6b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:11.271646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1163/2331] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:14:11.276012 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f036d59-37df-46db-8326-b02748513460 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:11.289543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f036d59-37df-46db-8326-b02748513460 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:11.295549 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f4045-8bdb-4625-9df9-e79d0442ee38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:11.296458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f4045-8bdb-4625-9df9-e79d0442ee38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:11.296744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f4045-8bdb-4625-9df9-e79d0442ee38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:11.297064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f4045-8bdb-4625-9df9-e79d0442ee38 admin admin] PATH_INFO: `/v3/projects/7aa60b10d8b94ba4a0eeb6cf8b752f01/users/df15aa9afbdc4e1d87dc3b535883745a/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:11.313711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f4045-8bdb-4625-9df9-e79d0442ee38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa60b10d8b94ba4a0eeb6cf8b752f01, user_id=df15aa9afbdc4e1d87dc3b535883745a, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:11.316416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f4045-8bdb-4625-9df9-e79d0442ee38 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:11.332414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1169/2332] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:14:11 2022] PUT /identity/v3/projects/7aa60b10d8b94ba4a0eeb6cf8b752f01/users/df15aa9afbdc4e1d87dc3b535883745a/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:11.337261 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475861b-25eb-49f4-ae70-dfed7bb44a6b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:11.348313 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475861b-25eb-49f4-ae70-dfed7bb44a6b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:11.417748 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b57dd4-bf6e-4b7a-a750-565feedeec81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:11.418611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57dd4-bf6e-4b7a-a750-565feedeec81 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:11.419092 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57dd4-bf6e-4b7a-a750-565feedeec81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:11.419232 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57dd4-bf6e-4b7a-a750-565feedeec81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:11.419973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b57dd4-bf6e-4b7a-a750-565feedeec81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:11.421665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b57dd4-bf6e-4b7a-a750-565feedeec81 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:11.433369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1164/2333] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:14:11.438291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f4045-8bdb-4625-9df9-e79d0442ee38 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:11.453380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f4045-8bdb-4625-9df9-e79d0442ee38 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:11.458548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2729b4-dbfe-4636-92a3-ecb61dbc5890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:11.459841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2729b4-dbfe-4636-92a3-ecb61dbc5890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:11.460127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2729b4-dbfe-4636-92a3-ecb61dbc5890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:11.460404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2729b4-dbfe-4636-92a3-ecb61dbc5890 admin admin] PATH_INFO: `/v3/projects/7aa60b10d8b94ba4a0eeb6cf8b752f01/users/df15aa9afbdc4e1d87dc3b535883745a/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:11.462855 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2729b4-dbfe-4636-92a3-ecb61dbc5890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa60b10d8b94ba4a0eeb6cf8b752f01, user_id=df15aa9afbdc4e1d87dc3b535883745a, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:11.467591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2729b4-dbfe-4636-92a3-ecb61dbc5890 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:11.495743 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ac832-fbd6-406e-9d4b-0036a92b11e3 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:11.496144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ac832-fbd6-406e-9d4b-0036a92b11e3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:11.496468 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ac832-fbd6-406e-9d4b-0036a92b11e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:11.498375 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1170/2334] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:14:11 2022] PUT /identity/v3/projects/7aa60b10d8b94ba4a0eeb6cf8b752f01/users/df15aa9afbdc4e1d87dc3b535883745a/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:14:11.534115 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-cd4ac832-fbd6-406e-9d4b-0036a92b11e3 None None] MFA Rules not processed for user `df15aa9afbdc4e1d87dc3b535883745a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:11.535395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-cd4ac832-fbd6-406e-9d4b-0036a92b11e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:11.594008 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:11.644254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1165/2335] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:11.652360 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2729b4-dbfe-4636-92a3-ecb61dbc5890 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:11.654353 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1c2729b4-dbfe-4636-92a3-ecb61dbc5890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:11.725159 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2729b4-dbfe-4636-92a3-ecb61dbc5890 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:11.729779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec1b8f-b660-439e-8f30-e6423327965d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:11.730632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec1b8f-b660-439e-8f30-e6423327965d service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:11.730917 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec1b8f-b660-439e-8f30-e6423327965d service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:11.731190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec1b8f-b660-439e-8f30-e6423327965d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:11.741745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec1b8f-b660-439e-8f30-e6423327965d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:11.742906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec1b8f-b660-439e-8f30-e6423327965d service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:11.754161 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1171/2336] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:11.829691 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ac832-fbd6-406e-9d4b-0036a92b11e3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:11.912646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ac832-fbd6-406e-9d4b-0036a92b11e3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:11.918744 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:11.919733 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:11.920105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:11.920455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] PATH_INFO: `/v3/users/743d67814a3f4ac69bc7dbca50ad834b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:11.921772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=743d67814a3f4ac69bc7dbca50ad834b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:11.923914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:11.990974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '743d67814a3f4ac69bc7dbca50ad834b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:11.991674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '743d67814a3f4ac69bc7dbca50ad834b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.000982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '743d67814a3f4ac69bc7dbca50ad834b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.015666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '743d67814a3f4ac69bc7dbca50ad834b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.016399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '743d67814a3f4ac69bc7dbca50ad834b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.030610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1166/2337] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:14:11 2022] DELETE /identity/v3/users/743d67814a3f4ac69bc7dbca50ad834b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:12.034935 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df7bf3-09af-43c8-8824-9661e6fa46ea None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:12.035242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df7bf3-09af-43c8-8824-9661e6fa46ea None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:12.035541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df7bf3-09af-43c8-8824-9661e6fa46ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:12.054463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-05df7bf3-09af-43c8-8824-9661e6fa46ea None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:12.055704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-05df7bf3-09af-43c8-8824-9661e6fa46ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:12.121986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1172/2338] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:12.129353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:12.136802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:12.203485 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebad67-4e15-4bcf-98de-1f98a7b5bb1e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:12.207452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa9ff8-0eaf-4a5b-8e60-79b57ff3bbac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:12.208424 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa9ff8-0eaf-4a5b-8e60-79b57ff3bbac service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:12.208845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa9ff8-0eaf-4a5b-8e60-79b57ff3bbac service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:12.209250 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa9ff8-0eaf-4a5b-8e60-79b57ff3bbac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:12.221478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa9ff8-0eaf-4a5b-8e60-79b57ff3bbac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:12.223305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa9ff8-0eaf-4a5b-8e60-79b57ff3bbac service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:12.234570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1167/2339] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:12.562930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df7bf3-09af-43c8-8824-9661e6fa46ea None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:12.564471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-05df7bf3-09af-43c8-8824-9661e6fa46ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:12.602896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df7bf3-09af-43c8-8824-9661e6fa46ea None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:12.610501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:12.611431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:12.611708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:12.611980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] PATH_INFO: `/v3/projects/78ac940025664893841da6ea5778863c` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:12.613399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ac940025664893841da6ea5778863c)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:12.615529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:12.639143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:12.686570 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ac940025664893841da6ea5778863c', 'request_id': 'req-142570d0-a7e2-494a-b5d7-68c390a36735'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.692667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ac940025664893841da6ea5778863c', 'request_id': 'req-142570d0-a7e2-494a-b5d7-68c390a36735'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.700853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ac940025664893841da6ea5778863c', 'request_id': 'req-142570d0-a7e2-494a-b5d7-68c390a36735'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.739976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] The token cache is being invalidate because project 78ac940025664893841da6ea5778863c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:14:12.740201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.740949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.742921 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1173/2340] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:14:12 2022] DELETE /identity/v3/projects/78ac940025664893841da6ea5778863c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:12.748176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa9ff8-0eaf-4a5b-8e60-79b57ff3bbac service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:12.755965 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:12.760294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa9ff8-0eaf-4a5b-8e60-79b57ff3bbac service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:12.843853 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1940fc-661e-44f1-b27f-95d2893d9a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:12.845280 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1940fc-661e-44f1-b27f-95d2893d9a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:12.845991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1940fc-661e-44f1-b27f-95d2893d9a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:12.846414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1940fc-661e-44f1-b27f-95d2893d9a7f admin admin] PATH_INFO: `/v3/users/df15aa9afbdc4e1d87dc3b535883745a` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:12.848053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1940fc-661e-44f1-b27f-95d2893d9a7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=df15aa9afbdc4e1d87dc3b535883745a)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:12.850106 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1940fc-661e-44f1-b27f-95d2893d9a7f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:12.909174 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-0f1940fc-661e-44f1-b27f-95d2893d9a7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df15aa9afbdc4e1d87dc3b535883745a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.910203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-0f1940fc-661e-44f1-b27f-95d2893d9a7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df15aa9afbdc4e1d87dc3b535883745a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.921099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-0f1940fc-661e-44f1-b27f-95d2893d9a7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df15aa9afbdc4e1d87dc3b535883745a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.937417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-0f1940fc-661e-44f1-b27f-95d2893d9a7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df15aa9afbdc4e1d87dc3b535883745a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.938676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-0f1940fc-661e-44f1-b27f-95d2893d9a7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df15aa9afbdc4e1d87dc3b535883745a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:12.955511 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1168/2341] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:14:12 2022] DELETE /identity/v3/users/df15aa9afbdc4e1d87dc3b535883745a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:13.279348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:13.280891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:13.306519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142570d0-a7e2-494a-b5d7-68c390a36735 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:13.376377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31b106-5047-408d-a672-bf85b64a317b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:13.377383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31b106-5047-408d-a672-bf85b64a317b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:13.377705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31b106-5047-408d-a672-bf85b64a317b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:13.378046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31b106-5047-408d-a672-bf85b64a317b admin admin] PATH_INFO: `/v3/projects/7aa60b10d8b94ba4a0eeb6cf8b752f01` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:13.379408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31b106-5047-408d-a672-bf85b64a317b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa60b10d8b94ba4a0eeb6cf8b752f01)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:13.381896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31b106-5047-408d-a672-bf85b64a317b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:13.443204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7d31b106-5047-408d-a672-bf85b64a317b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa60b10d8b94ba4a0eeb6cf8b752f01', 'request_id': 'req-7d31b106-5047-408d-a672-bf85b64a317b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:13.449167 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7d31b106-5047-408d-a672-bf85b64a317b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa60b10d8b94ba4a0eeb6cf8b752f01', 'request_id': 'req-7d31b106-5047-408d-a672-bf85b64a317b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:13.455814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7d31b106-5047-408d-a672-bf85b64a317b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa60b10d8b94ba4a0eeb6cf8b752f01', 'request_id': 'req-7d31b106-5047-408d-a672-bf85b64a317b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:13.470349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7d31b106-5047-408d-a672-bf85b64a317b admin admin] The token cache is being invalidate because project 7aa60b10d8b94ba4a0eeb6cf8b752f01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:14:13.470649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7d31b106-5047-408d-a672-bf85b64a317b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:13.471313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7d31b106-5047-408d-a672-bf85b64a317b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:13.473016 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1174/2342] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:14:13 2022] DELETE /identity/v3/projects/7aa60b10d8b94ba4a0eeb6cf8b752f01 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:13.478449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250acf1b-11e4-4415-aa64-11a26a469577 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:13.478937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250acf1b-11e4-4415-aa64-11a26a469577 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:13.479035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250acf1b-11e4-4415-aa64-11a26a469577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:13.500372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-250acf1b-11e4-4415-aa64-11a26a469577 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:13.501578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-250acf1b-11e4-4415-aa64-11a26a469577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:13.563095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1169/2343] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:13.570700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256ce6b-c6f6-42ef-88f8-6385c2ff7098 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:13.570993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256ce6b-c6f6-42ef-88f8-6385c2ff7098 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:13.571272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256ce6b-c6f6-42ef-88f8-6385c2ff7098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:13.591728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e256ce6b-c6f6-42ef-88f8-6385c2ff7098 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:13.592934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e256ce6b-c6f6-42ef-88f8-6385c2ff7098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:13.601677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1175/2344] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:13.606699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250acf1b-11e4-4415-aa64-11a26a469577 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:13.617095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250acf1b-11e4-4415-aa64-11a26a469577 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:13.621933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc1fa7-24f8-4f84-86fd-779f73d5912f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:13.622582 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc1fa7-24f8-4f84-86fd-779f73d5912f admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:13.622936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc1fa7-24f8-4f84-86fd-779f73d5912f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:13.623211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc1fa7-24f8-4f84-86fd-779f73d5912f admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:13.623933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc1fa7-24f8-4f84-86fd-779f73d5912f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:14:13.624234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc1fa7-24f8-4f84-86fd-779f73d5912f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:13.625859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc1fa7-24f8-4f84-86fd-779f73d5912f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:13.638555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1170/2345] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:14:13.643566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256ce6b-c6f6-42ef-88f8-6385c2ff7098 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:13.656301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256ce6b-c6f6-42ef-88f8-6385c2ff7098 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:13.662295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7ef8b-59b1-485a-8016-ac8d0d661b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:13.662951 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ef8b-59b1-485a-8016-ac8d0d661b23 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:13.663231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ef8b-59b1-485a-8016-ac8d0d661b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:13.663513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ef8b-59b1-485a-8016-ac8d0d661b23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:13.664205 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7ef8b-59b1-485a-8016-ac8d0d661b23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:13.666156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7ef8b-59b1-485a-8016-ac8d0d661b23 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:13.694705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1176/2346] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:14:13 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:14:13.699222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc1fa7-24f8-4f84-86fd-779f73d5912f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:13.709952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc1fa7-24f8-4f84-86fd-779f73d5912f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:13.718862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:13.771436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096fdb8-b7d1-489e-8b88-ed8c8aa7c3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:13.772113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096fdb8-b7d1-489e-8b88-ed8c8aa7c3ae admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:13.772372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096fdb8-b7d1-489e-8b88-ed8c8aa7c3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:13.772649 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096fdb8-b7d1-489e-8b88-ed8c8aa7c3ae admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:13.773348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096fdb8-b7d1-489e-8b88-ed8c8aa7c3ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:13.775344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096fdb8-b7d1-489e-8b88-ed8c8aa7c3ae admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:13.798819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1171/2347] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:14:13 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:14:13.805263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7ef8b-59b1-485a-8016-ac8d0d661b23 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:13.819073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7ef8b-59b1-485a-8016-ac8d0d661b23 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:13.825109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a72af-a847-4ec1-9986-3fa861f2bac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:13.825823 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a72af-a847-4ec1-9986-3fa861f2bac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:13.826108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a72af-a847-4ec1-9986-3fa861f2bac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:13.826389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a72af-a847-4ec1-9986-3fa861f2bac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:13.827123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a72af-a847-4ec1-9986-3fa861f2bac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:13.828759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a72af-a847-4ec1-9986-3fa861f2bac2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:13.830153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:13.842509 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1177/2348] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:14:13.847119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096fdb8-b7d1-489e-8b88-ed8c8aa7c3ae admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:13.857831 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096fdb8-b7d1-489e-8b88-ed8c8aa7c3ae admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:13.863085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c22fb-fd2d-4bc8-a585-1a855de84e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:13.863752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c22fb-fd2d-4bc8-a585-1a855de84e08 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:13.864034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c22fb-fd2d-4bc8-a585-1a855de84e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:13.864311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c22fb-fd2d-4bc8-a585-1a855de84e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:13.865040 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c22fb-fd2d-4bc8-a585-1a855de84e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:13.866709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c22fb-fd2d-4bc8-a585-1a855de84e08 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:13.877458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1172/2349] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:14:13.881796 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a72af-a847-4ec1-9986-3fa861f2bac2 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:13.892268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a72af-a847-4ec1-9986-3fa861f2bac2 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:13.897331 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83aa6a6-e7f6-48c3-8e7c-f7e8470da531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:13.898664 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83aa6a6-e7f6-48c3-8e7c-f7e8470da531 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:13.899130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83aa6a6-e7f6-48c3-8e7c-f7e8470da531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:13.899614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83aa6a6-e7f6-48c3-8e7c-f7e8470da531 admin admin] PATH_INFO: `/v3/projects/7d19f8464d324c4fb962b2196332e624/users/cbff382df39c48edbb2621b6fcafa84c/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:13.915824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83aa6a6-e7f6-48c3-8e7c-f7e8470da531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d19f8464d324c4fb962b2196332e624, user_id=cbff382df39c48edbb2621b6fcafa84c, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:13.918742 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83aa6a6-e7f6-48c3-8e7c-f7e8470da531 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:13.933926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1178/2350] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:14:13 2022] PUT /identity/v3/projects/7d19f8464d324c4fb962b2196332e624/users/cbff382df39c48edbb2621b6fcafa84c/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:13.946727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254961ff-6ade-437b-8598-1a8dece756cc None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:13.947987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254961ff-6ade-437b-8598-1a8dece756cc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:13.947987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254961ff-6ade-437b-8598-1a8dece756cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:13.991542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-254961ff-6ade-437b-8598-1a8dece756cc None None] MFA Rules not processed for user `cbff382df39c48edbb2621b6fcafa84c`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:13.992806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-254961ff-6ade-437b-8598-1a8dece756cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:14.071732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1173/2351] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:14.076804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83aa6a6-e7f6-48c3-8e7c-f7e8470da531 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:14.150004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83aa6a6-e7f6-48c3-8e7c-f7e8470da531 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:14.156694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba2239-40ac-47f1-aeb8-d922f3283147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:14.157505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba2239-40ac-47f1-aeb8-d922f3283147 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:14.157908 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba2239-40ac-47f1-aeb8-d922f3283147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:14.158281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba2239-40ac-47f1-aeb8-d922f3283147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:14.159090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba2239-40ac-47f1-aeb8-d922f3283147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:14.161168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba2239-40ac-47f1-aeb8-d922f3283147 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:14.190057 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1179/2352] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:14:14 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:14:14.194200 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254961ff-6ade-437b-8598-1a8dece756cc None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:14.205485 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254961ff-6ade-437b-8598-1a8dece756cc None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:14.273055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c09b3-83d7-4c5b-a51b-8b649537e095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:14.274029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c09b3-83d7-4c5b-a51b-8b649537e095 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:14.274340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c09b3-83d7-4c5b-a51b-8b649537e095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:14.274621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c09b3-83d7-4c5b-a51b-8b649537e095 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:14.275364 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c09b3-83d7-4c5b-a51b-8b649537e095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:14.278218 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c09b3-83d7-4c5b-a51b-8b649537e095 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:14.307171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1174/2353] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:14:14 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:14:14.308286 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba2239-40ac-47f1-aeb8-d922f3283147 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:14.320629 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba2239-40ac-47f1-aeb8-d922f3283147 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:14.326876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b585551-bb28-48f6-8032-9b602632f282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:14.327558 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b585551-bb28-48f6-8032-9b602632f282 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:14.327867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b585551-bb28-48f6-8032-9b602632f282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:14.328144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b585551-bb28-48f6-8032-9b602632f282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:14.328871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b585551-bb28-48f6-8032-9b602632f282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:14.330544 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b585551-bb28-48f6-8032-9b602632f282 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:14.342636 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1180/2354] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:14:14.346873 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c09b3-83d7-4c5b-a51b-8b649537e095 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:14.357287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c09b3-83d7-4c5b-a51b-8b649537e095 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:14.362265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec5089-128d-4480-91d1-ce1e25e7f86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:14.363159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec5089-128d-4480-91d1-ce1e25e7f86b admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:14.363439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec5089-128d-4480-91d1-ce1e25e7f86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:14.363739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec5089-128d-4480-91d1-ce1e25e7f86b admin admin] PATH_INFO: `/v3/projects/4bae1bf4dad0493783c7d7862a628e01/users/6f48b7ea5eb94370a40aa97962436e72/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:14.367099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:14.380713 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec5089-128d-4480-91d1-ce1e25e7f86b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bae1bf4dad0493783c7d7862a628e01, user_id=6f48b7ea5eb94370a40aa97962436e72, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:14.383446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec5089-128d-4480-91d1-ce1e25e7f86b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:14.398509 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1175/2355] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:14:14 2022] PUT /identity/v3/projects/4bae1bf4dad0493783c7d7862a628e01/users/6f48b7ea5eb94370a40aa97962436e72/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:14:14.402944 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b585551-bb28-48f6-8032-9b602632f282 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:14.413306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b585551-bb28-48f6-8032-9b602632f282 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:14.472425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221402dc-cbee-4274-b713-74dc51f3ff29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:14.473098 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221402dc-cbee-4274-b713-74dc51f3ff29 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:14.473373 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221402dc-cbee-4274-b713-74dc51f3ff29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:14.473681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221402dc-cbee-4274-b713-74dc51f3ff29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:14.474406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221402dc-cbee-4274-b713-74dc51f3ff29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:14.476028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221402dc-cbee-4274-b713-74dc51f3ff29 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:14.487834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1181/2356] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:14:14.492082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec5089-128d-4480-91d1-ce1e25e7f86b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:14.504061 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec5089-128d-4480-91d1-ce1e25e7f86b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:14.508987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20807c-303f-41a3-83cd-74e8e14c0b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:14.509919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20807c-303f-41a3-83cd-74e8e14c0b4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:14.510198 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20807c-303f-41a3-83cd-74e8e14c0b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:14.510516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20807c-303f-41a3-83cd-74e8e14c0b4f admin admin] PATH_INFO: `/v3/projects/4bae1bf4dad0493783c7d7862a628e01/users/6f48b7ea5eb94370a40aa97962436e72/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:14.512631 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20807c-303f-41a3-83cd-74e8e14c0b4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bae1bf4dad0493783c7d7862a628e01, user_id=6f48b7ea5eb94370a40aa97962436e72, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:14.515317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20807c-303f-41a3-83cd-74e8e14c0b4f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:14.526738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1176/2357] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:14:14 2022] PUT /identity/v3/projects/4bae1bf4dad0493783c7d7862a628e01/users/6f48b7ea5eb94370a40aa97962436e72/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:14.536991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e993885-57d3-4747-af07-ac8ea26b03e4 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:14.537282 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e993885-57d3-4747-af07-ac8ea26b03e4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:14.537555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e993885-57d3-4747-af07-ac8ea26b03e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:14.567344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-0e993885-57d3-4747-af07-ac8ea26b03e4 None None] MFA Rules not processed for user `6f48b7ea5eb94370a40aa97962436e72`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:14.568576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0e993885-57d3-4747-af07-ac8ea26b03e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:14.607517 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:14.651536 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1182/2358] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:14.663553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20807c-303f-41a3-83cd-74e8e14c0b4f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:14.665742 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ea20807c-303f-41a3-83cd-74e8e14c0b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:14.726632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20807c-303f-41a3-83cd-74e8e14c0b4f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:14.730576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5974054-6ee1-418e-8bbf-cf24a1d3baed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:14.731196 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5974054-6ee1-418e-8bbf-cf24a1d3baed service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:14.731508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5974054-6ee1-418e-8bbf-cf24a1d3baed service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:14.731788 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5974054-6ee1-418e-8bbf-cf24a1d3baed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:14.800078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5974054-6ee1-418e-8bbf-cf24a1d3baed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:14.801535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5974054-6ee1-418e-8bbf-cf24a1d3baed service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:14.811361 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1177/2359] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:16.775706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e993885-57d3-4747-af07-ac8ea26b03e4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:16.777891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0e993885-57d3-4747-af07-ac8ea26b03e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:16.865124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e993885-57d3-4747-af07-ac8ea26b03e4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:16.870494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539963f3-8c63-4e12-80d6-76675b1b44b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:16.871176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539963f3-8c63-4e12-80d6-76675b1b44b5 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:16.871464 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539963f3-8c63-4e12-80d6-76675b1b44b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:16.871794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539963f3-8c63-4e12-80d6-76675b1b44b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:16.873528 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-539963f3-8c63-4e12-80d6-76675b1b44b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:16.885349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539963f3-8c63-4e12-80d6-76675b1b44b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:16.887301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539963f3-8c63-4e12-80d6-76675b1b44b5 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:16.897475 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1183/2360] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:18.704487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5974054-6ee1-418e-8bbf-cf24a1d3baed service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:18.706164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e5974054-6ee1-418e-8bbf-cf24a1d3baed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:18.787976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5974054-6ee1-418e-8bbf-cf24a1d3baed service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:18.793766 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d54889-c1cf-42f5-b493-9fac5774f728 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:18.794421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d54889-c1cf-42f5-b493-9fac5774f728 service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:18.794708 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d54889-c1cf-42f5-b493-9fac5774f728 service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:18.794993 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d54889-c1cf-42f5-b493-9fac5774f728 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:18.796802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-66d54889-c1cf-42f5-b493-9fac5774f728 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:18.808583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d54889-c1cf-42f5-b493-9fac5774f728 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:18.810248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d54889-c1cf-42f5-b493-9fac5774f728 service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:18.820658 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1178/2361] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:19.059741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539963f3-8c63-4e12-80d6-76675b1b44b5 service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:19.070733 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539963f3-8c63-4e12-80d6-76675b1b44b5 service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:19.074474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6fd3f-eed2-4a47-9b6a-f08b51a31ed5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:19.075128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6fd3f-eed2-4a47-9b6a-f08b51a31ed5 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:19.075426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6fd3f-eed2-4a47-9b6a-f08b51a31ed5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:19.075708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6fd3f-eed2-4a47-9b6a-f08b51a31ed5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:19.085294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6fd3f-eed2-4a47-9b6a-f08b51a31ed5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:19.086830 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6fd3f-eed2-4a47-9b6a-f08b51a31ed5 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:19.097682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1184/2362] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:20.454348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41caaf18-a06a-4929-a745-a3e1caa08a2e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:20.454807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41caaf18-a06a-4929-a745-a3e1caa08a2e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:20.455013 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41caaf18-a06a-4929-a745-a3e1caa08a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:20.473941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-41caaf18-a06a-4929-a745-a3e1caa08a2e None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:20.475206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-41caaf18-a06a-4929-a745-a3e1caa08a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:20.485594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:20.537119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1179/2363] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:14:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:20.549189 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6fd3f-eed2-4a47-9b6a-f08b51a31ed5 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:20.550626 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-68a6fd3f-eed2-4a47-9b6a-f08b51a31ed5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:20.623922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6fd3f-eed2-4a47-9b6a-f08b51a31ed5 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:20.631390 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba336f-3077-4ad8-be08-58e6091d421a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:20.632126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba336f-3077-4ad8-be08-58e6091d421a service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:20.632514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba336f-3077-4ad8-be08-58e6091d421a service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:20.632852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba336f-3077-4ad8-be08-58e6091d421a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:20.643840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba336f-3077-4ad8-be08-58e6091d421a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:20.645277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba336f-3077-4ad8-be08-58e6091d421a service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:20.654409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1185/2364] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:14:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 10:14:24.025049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41caaf18-a06a-4929-a745-a3e1caa08a2e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:24.105028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41caaf18-a06a-4929-a745-a3e1caa08a2e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:24.109752 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:24.110640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:24.110952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:24.111198 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] PATH_INFO: `/v3/users/cbff382df39c48edbb2621b6fcafa84c` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:24.112381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbff382df39c48edbb2621b6fcafa84c)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:24.114373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:24.172602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbff382df39c48edbb2621b6fcafa84c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:24.173309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbff382df39c48edbb2621b6fcafa84c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:24.183458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbff382df39c48edbb2621b6fcafa84c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:24.196862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbff382df39c48edbb2621b6fcafa84c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:24.197659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbff382df39c48edbb2621b6fcafa84c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:24.210519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1180/2365] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:14:24 2022] DELETE /identity/v3/users/cbff382df39c48edbb2621b6fcafa84c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:24.215803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78754bf9-6ba3-4fc1-8749-d9602f6da38d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:24.216311 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78754bf9-6ba3-4fc1-8749-d9602f6da38d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:24.216419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78754bf9-6ba3-4fc1-8749-d9602f6da38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:24.234355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-78754bf9-6ba3-4fc1-8749-d9602f6da38d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:24.235565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-78754bf9-6ba3-4fc1-8749-d9602f6da38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:24.261333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:24.298567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1186/2366] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:14:24.309312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:24.310832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:24.361670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:24.380272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ba664-b163-4118-bfc5-853a373238b2 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:24.384139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97607d15-63f4-4e68-9ecf-d6868bba2797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:24.384838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607d15-63f4-4e68-9ecf-d6868bba2797 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:24.385102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607d15-63f4-4e68-9ecf-d6868bba2797 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:24.386373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607d15-63f4-4e68-9ecf-d6868bba2797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:24.396680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97607d15-63f4-4e68-9ecf-d6868bba2797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:24.398302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97607d15-63f4-4e68-9ecf-d6868bba2797 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:24.410153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1181/2367] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:14:24.580979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78754bf9-6ba3-4fc1-8749-d9602f6da38d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:24.582431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-78754bf9-6ba3-4fc1-8749-d9602f6da38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:24.720592 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78754bf9-6ba3-4fc1-8749-d9602f6da38d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:24.727151 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862580ba-b4f7-4612-8cb9-d48c1644faef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:24.727914 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862580ba-b4f7-4612-8cb9-d48c1644faef service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:24.728263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862580ba-b4f7-4612-8cb9-d48c1644faef service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:24.728612 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862580ba-b4f7-4612-8cb9-d48c1644faef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:24.730587 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-862580ba-b4f7-4612-8cb9-d48c1644faef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:24.807668 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862580ba-b4f7-4612-8cb9-d48c1644faef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:24.809609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862580ba-b4f7-4612-8cb9-d48c1644faef service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:24.829185 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1187/2368] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:24.876274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97607d15-63f4-4e68-9ecf-d6868bba2797 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:24.876274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-97607d15-63f4-4e68-9ecf-d6868bba2797 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:24.895676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97607d15-63f4-4e68-9ecf-d6868bba2797 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:24.901692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:24.902942 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:24.903226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:24.903508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] PATH_INFO: `/v3/projects/7d19f8464d324c4fb962b2196332e624` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:24.905384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d19f8464d324c4fb962b2196332e624)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:24.907361 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:25.025961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d19f8464d324c4fb962b2196332e624', 'request_id': 'req-44ecd225-0366-495e-ad5a-d2f49ac8b02c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.034307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d19f8464d324c4fb962b2196332e624', 'request_id': 'req-44ecd225-0366-495e-ad5a-d2f49ac8b02c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.040787 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d19f8464d324c4fb962b2196332e624', 'request_id': 'req-44ecd225-0366-495e-ad5a-d2f49ac8b02c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.065086 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] The token cache is being invalidate because project 7d19f8464d324c4fb962b2196332e624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:14:25.065414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.066580 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.068982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1182/2369] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:14:24 2022] DELETE /identity/v3/projects/7d19f8464d324c4fb962b2196332e624 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:25.077376 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862580ba-b4f7-4612-8cb9-d48c1644faef service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:25.091923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862580ba-b4f7-4612-8cb9-d48c1644faef service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:25.173325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444a818-c646-49a2-9e2d-750bd0633b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:25.174270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444a818-c646-49a2-9e2d-750bd0633b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:25.174551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444a818-c646-49a2-9e2d-750bd0633b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:25.174831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444a818-c646-49a2-9e2d-750bd0633b29 admin admin] PATH_INFO: `/v3/users/6f48b7ea5eb94370a40aa97962436e72` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:25.176054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444a818-c646-49a2-9e2d-750bd0633b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f48b7ea5eb94370a40aa97962436e72)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:25.178067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444a818-c646-49a2-9e2d-750bd0633b29 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:25.187723 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:25.265447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e444a818-c646-49a2-9e2d-750bd0633b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f48b7ea5eb94370a40aa97962436e72'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.266215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e444a818-c646-49a2-9e2d-750bd0633b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f48b7ea5eb94370a40aa97962436e72'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.288752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e444a818-c646-49a2-9e2d-750bd0633b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f48b7ea5eb94370a40aa97962436e72'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.311257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e444a818-c646-49a2-9e2d-750bd0633b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f48b7ea5eb94370a40aa97962436e72'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.313477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e444a818-c646-49a2-9e2d-750bd0633b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f48b7ea5eb94370a40aa97962436e72'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.343780 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1188/2370] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:14:25 2022] DELETE /identity/v3/users/6f48b7ea5eb94370a40aa97962436e72 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:25.730968 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:25.732690 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:25.755481 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecd225-0366-495e-ad5a-d2f49ac8b02c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:25.833501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876332b1-16b9-4da4-ac75-49b4ef78693c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:25.834649 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876332b1-16b9-4da4-ac75-49b4ef78693c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:25.834935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876332b1-16b9-4da4-ac75-49b4ef78693c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:25.835211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876332b1-16b9-4da4-ac75-49b4ef78693c admin admin] PATH_INFO: `/v3/projects/4bae1bf4dad0493783c7d7862a628e01` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:25.836417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876332b1-16b9-4da4-ac75-49b4ef78693c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bae1bf4dad0493783c7d7862a628e01)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:25.838451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876332b1-16b9-4da4-ac75-49b4ef78693c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:25.893361 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-876332b1-16b9-4da4-ac75-49b4ef78693c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bae1bf4dad0493783c7d7862a628e01', 'request_id': 'req-876332b1-16b9-4da4-ac75-49b4ef78693c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.900033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-876332b1-16b9-4da4-ac75-49b4ef78693c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bae1bf4dad0493783c7d7862a628e01', 'request_id': 'req-876332b1-16b9-4da4-ac75-49b4ef78693c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.907020 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-876332b1-16b9-4da4-ac75-49b4ef78693c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bae1bf4dad0493783c7d7862a628e01', 'request_id': 'req-876332b1-16b9-4da4-ac75-49b4ef78693c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.911859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:25.929478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-876332b1-16b9-4da4-ac75-49b4ef78693c admin admin] The token cache is being invalidate because project 4bae1bf4dad0493783c7d7862a628e01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:14:25.930047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-876332b1-16b9-4da4-ac75-49b4ef78693c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.930888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-876332b1-16b9-4da4-ac75-49b4ef78693c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:25.932862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1183/2371] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:14:25 2022] DELETE /identity/v3/projects/4bae1bf4dad0493783c7d7862a628e01 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:25.939688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d01a7-2a4b-41e8-a4df-5967428375c2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:25.940092 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d01a7-2a4b-41e8-a4df-5967428375c2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:25.940477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d01a7-2a4b-41e8-a4df-5967428375c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:25.963534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-0f4d01a7-2a4b-41e8-a4df-5967428375c2 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:25.965925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0f4d01a7-2a4b-41e8-a4df-5967428375c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:26.043107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1189/2372] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:26.052702 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05b657-dde0-436e-97cd-1d48534a8bc7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:26.053259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05b657-dde0-436e-97cd-1d48534a8bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:26.053259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05b657-dde0-436e-97cd-1d48534a8bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:26.074140 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2f05b657-dde0-436e-97cd-1d48534a8bc7 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:26.075865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2f05b657-dde0-436e-97cd-1d48534a8bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:26.087961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1184/2373] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:26.089690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d01a7-2a4b-41e8-a4df-5967428375c2 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:26.100815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d01a7-2a4b-41e8-a4df-5967428375c2 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:26.106012 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc33f00-93d5-4a85-b722-6f4236a04615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:26.106801 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc33f00-93d5-4a85-b722-6f4236a04615 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:26.107187 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc33f00-93d5-4a85-b722-6f4236a04615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:26.107670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc33f00-93d5-4a85-b722-6f4236a04615 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:26.108514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc33f00-93d5-4a85-b722-6f4236a04615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:14:26.109056 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc33f00-93d5-4a85-b722-6f4236a04615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:26.111038 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc33f00-93d5-4a85-b722-6f4236a04615 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:26.124316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1190/2374] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:14:26.128648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05b657-dde0-436e-97cd-1d48534a8bc7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:26.139189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05b657-dde0-436e-97cd-1d48534a8bc7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:26.144039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cebbd5-90e7-4bd7-a90c-4e8dcf6ead4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:26.144651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cebbd5-90e7-4bd7-a90c-4e8dcf6ead4e admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:26.144935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cebbd5-90e7-4bd7-a90c-4e8dcf6ead4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:26.145212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cebbd5-90e7-4bd7-a90c-4e8dcf6ead4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:26.145981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cebbd5-90e7-4bd7-a90c-4e8dcf6ead4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:26.148064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cebbd5-90e7-4bd7-a90c-4e8dcf6ead4e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:26.180297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1185/2375] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:14:26 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:14:26.184568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc33f00-93d5-4a85-b722-6f4236a04615 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:26.195166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc33f00-93d5-4a85-b722-6f4236a04615 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:26.258482 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fef6b5-ab43-41e6-b717-5359c941fcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:26.259199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fef6b5-ab43-41e6-b717-5359c941fcbe admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:26.259490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fef6b5-ab43-41e6-b717-5359c941fcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:26.259765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fef6b5-ab43-41e6-b717-5359c941fcbe admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:26.260556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fef6b5-ab43-41e6-b717-5359c941fcbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:26.262538 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fef6b5-ab43-41e6-b717-5359c941fcbe admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:26.290766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1191/2376] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:14:26 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:14:26.292801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cebbd5-90e7-4bd7-a90c-4e8dcf6ead4e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:26.303777 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cebbd5-90e7-4bd7-a90c-4e8dcf6ead4e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:26.309255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d7aab-003f-4ff1-aee6-3c22bc1b45d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:26.310045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d7aab-003f-4ff1-aee6-3c22bc1b45d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:26.310457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d7aab-003f-4ff1-aee6-3c22bc1b45d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:26.310813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d7aab-003f-4ff1-aee6-3c22bc1b45d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:26.311657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d7aab-003f-4ff1-aee6-3c22bc1b45d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:26.313576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d7aab-003f-4ff1-aee6-3c22bc1b45d5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:26.325914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1186/2377] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:14:26.330077 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fef6b5-ab43-41e6-b717-5359c941fcbe admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:26.335596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:26.342584 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fef6b5-ab43-41e6-b717-5359c941fcbe admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:26.349459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6c770-4dc2-442a-b6f5-12c77d4ea6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:26.350342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6c770-4dc2-442a-b6f5-12c77d4ea6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:26.350790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6c770-4dc2-442a-b6f5-12c77d4ea6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:26.351061 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6c770-4dc2-442a-b6f5-12c77d4ea6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:26.352211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6c770-4dc2-442a-b6f5-12c77d4ea6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:26.354434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6c770-4dc2-442a-b6f5-12c77d4ea6b4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:26.366190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1192/2378] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:14:26.370658 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d7aab-003f-4ff1-aee6-3c22bc1b45d5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:26.381920 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d7aab-003f-4ff1-aee6-3c22bc1b45d5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:26.387206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3ea5e-4324-439b-b2cb-21ce5b6b588a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:26.388147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3ea5e-4324-439b-b2cb-21ce5b6b588a admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:26.388429 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3ea5e-4324-439b-b2cb-21ce5b6b588a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:26.388736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3ea5e-4324-439b-b2cb-21ce5b6b588a admin admin] PATH_INFO: `/v3/projects/3bc7f3a6bdff404ba5960de029aed28d/users/868ae0bf2db14ba493df72eb727bec75/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:26.405945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3ea5e-4324-439b-b2cb-21ce5b6b588a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc7f3a6bdff404ba5960de029aed28d, user_id=868ae0bf2db14ba493df72eb727bec75, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:26.408906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3ea5e-4324-439b-b2cb-21ce5b6b588a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:26.436686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1187/2379] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:14:26 2022] PUT /identity/v3/projects/3bc7f3a6bdff404ba5960de029aed28d/users/868ae0bf2db14ba493df72eb727bec75/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:14:26.448137 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb5ed3-3782-44c5-a393-606cbdfef36f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:26.448431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb5ed3-3782-44c5-a393-606cbdfef36f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:26.448713 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb5ed3-3782-44c5-a393-606cbdfef36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:26.486388 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6edb5ed3-3782-44c5-a393-606cbdfef36f None None] MFA Rules not processed for user `868ae0bf2db14ba493df72eb727bec75`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:26.487620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6edb5ed3-3782-44c5-a393-606cbdfef36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:26.625363 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1193/2380] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:26.633137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3ea5e-4324-439b-b2cb-21ce5b6b588a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:26.635428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-41d3ea5e-4324-439b-b2cb-21ce5b6b588a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:26.732851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3ea5e-4324-439b-b2cb-21ce5b6b588a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:26.737190 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c9234-75f9-4320-83f3-4e799bf8c3bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:26.737951 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c9234-75f9-4320-83f3-4e799bf8c3bc service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:26.738242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c9234-75f9-4320-83f3-4e799bf8c3bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:26.738521 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c9234-75f9-4320-83f3-4e799bf8c3bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:26.748199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c9234-75f9-4320-83f3-4e799bf8c3bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:26.749676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c9234-75f9-4320-83f3-4e799bf8c3bc service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:26.759205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1188/2381] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:34.404450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da71368-7ac6-4a38-bc77-dcc6dda82f08 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:34.404956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da71368-7ac6-4a38-bc77-dcc6dda82f08 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:34.405220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da71368-7ac6-4a38-bc77-dcc6dda82f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:34.423999 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7da71368-7ac6-4a38-bc77-dcc6dda82f08 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:34.425351 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7da71368-7ac6-4a38-bc77-dcc6dda82f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:34.483762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1194/2382] 10.222.0.79 () {58 vars in 967 bytes} [Mon Jul 4 10:14:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:34.496395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c9234-75f9-4320-83f3-4e799bf8c3bc service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:34.498290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-949c9234-75f9-4320-83f3-4e799bf8c3bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:34.570870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c9234-75f9-4320-83f3-4e799bf8c3bc service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:34.576229 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74fefd-81d7-45c8-a772-54448ecaea8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:34.576879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74fefd-81d7-45c8-a772-54448ecaea8d service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:34.577167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74fefd-81d7-45c8-a772-54448ecaea8d service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:34.577447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74fefd-81d7-45c8-a772-54448ecaea8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:34.585727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74fefd-81d7-45c8-a772-54448ecaea8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:34.587316 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74fefd-81d7-45c8-a772-54448ecaea8d service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:34.595914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1189/2383] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:14:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 10:14:36.897752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da71368-7ac6-4a38-bc77-dcc6dda82f08 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:36.970016 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da71368-7ac6-4a38-bc77-dcc6dda82f08 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:36.974752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:36.975644 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:36.975926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:36.976204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] PATH_INFO: `/v3/users/868ae0bf2db14ba493df72eb727bec75` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:36.977410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=868ae0bf2db14ba493df72eb727bec75)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:36.979411 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:37.011983 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:37.032778 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868ae0bf2db14ba493df72eb727bec75'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:37.033453 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868ae0bf2db14ba493df72eb727bec75'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:37.042795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868ae0bf2db14ba493df72eb727bec75'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:37.056197 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868ae0bf2db14ba493df72eb727bec75'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:37.057015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868ae0bf2db14ba493df72eb727bec75'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:37.072189 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1195/2384] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:14:36 2022] DELETE /identity/v3/users/868ae0bf2db14ba493df72eb727bec75 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:37.077439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ceec5-5a3e-4542-a937-4db0f3abd288 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:37.078023 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ceec5-5a3e-4542-a937-4db0f3abd288 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:37.078023 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ceec5-5a3e-4542-a937-4db0f3abd288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:37.096833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7f4ceec5-5a3e-4542-a937-4db0f3abd288 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:37.098194 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7f4ceec5-5a3e-4542-a937-4db0f3abd288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:37.120731 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:37.172457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1190/2385] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:14:37.183438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:37.185721 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:37.247624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0b5a7-1dbd-4fbb-8e2b-a357bf1cd0ac admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:37.252035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e9e9d-c407-426e-a079-087b775c05fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:37.252698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e9e9d-c407-426e-a079-087b775c05fa service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:37.252980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e9e9d-c407-426e-a079-087b775c05fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:37.253250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e9e9d-c407-426e-a079-087b775c05fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:37.262902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e9e9d-c407-426e-a079-087b775c05fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:37.264440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e9e9d-c407-426e-a079-087b775c05fa service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:37.274023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1196/2386] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:37.554094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ceec5-5a3e-4542-a937-4db0f3abd288 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:37.555677 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7f4ceec5-5a3e-4542-a937-4db0f3abd288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:37.570170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ceec5-5a3e-4542-a937-4db0f3abd288 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:37.574999 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38a3b4-e1e2-400b-83ba-a3853618144c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:37.575866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38a3b4-e1e2-400b-83ba-a3853618144c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:37.576153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38a3b4-e1e2-400b-83ba-a3853618144c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:37.576583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38a3b4-e1e2-400b-83ba-a3853618144c admin admin] PATH_INFO: `/v3/projects/3bc7f3a6bdff404ba5960de029aed28d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:37.577818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38a3b4-e1e2-400b-83ba-a3853618144c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc7f3a6bdff404ba5960de029aed28d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:37.579838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38a3b4-e1e2-400b-83ba-a3853618144c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:37.628781 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-dd38a3b4-e1e2-400b-83ba-a3853618144c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc7f3a6bdff404ba5960de029aed28d', 'request_id': 'req-dd38a3b4-e1e2-400b-83ba-a3853618144c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:37.635487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-dd38a3b4-e1e2-400b-83ba-a3853618144c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc7f3a6bdff404ba5960de029aed28d', 'request_id': 'req-dd38a3b4-e1e2-400b-83ba-a3853618144c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:37.641269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-dd38a3b4-e1e2-400b-83ba-a3853618144c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc7f3a6bdff404ba5960de029aed28d', 'request_id': 'req-dd38a3b4-e1e2-400b-83ba-a3853618144c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:37.655830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-dd38a3b4-e1e2-400b-83ba-a3853618144c admin admin] The token cache is being invalidate because project 3bc7f3a6bdff404ba5960de029aed28d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:14:37.656126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-dd38a3b4-e1e2-400b-83ba-a3853618144c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:37.656774 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-dd38a3b4-e1e2-400b-83ba-a3853618144c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:14:37.658478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1191/2387] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:14:37 2022] DELETE /identity/v3/projects/3bc7f3a6bdff404ba5960de029aed28d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:37.663516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d023c-6cd0-48cb-a51a-c3c3f074b6c8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:37.663923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d023c-6cd0-48cb-a51a-c3c3f074b6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:37.664083 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d023c-6cd0-48cb-a51a-c3c3f074b6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:37.681953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4c7d023c-6cd0-48cb-a51a-c3c3f074b6c8 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:37.683201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4c7d023c-6cd0-48cb-a51a-c3c3f074b6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:37.745784 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1197/2388] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:37.753934 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22845b24-7b89-427b-b342-0b977414a18b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:37.754228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22845b24-7b89-427b-b342-0b977414a18b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:37.754511 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22845b24-7b89-427b-b342-0b977414a18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:37.773402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-22845b24-7b89-427b-b342-0b977414a18b None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:37.774634 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-22845b24-7b89-427b-b342-0b977414a18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:37.783124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1192/2389] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:14:37.787465 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d023c-6cd0-48cb-a51a-c3c3f074b6c8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:37.798028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d023c-6cd0-48cb-a51a-c3c3f074b6c8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:37.802734 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463897e3-7a8a-4a21-83c9-6e56348c7d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:37.803318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463897e3-7a8a-4a21-83c9-6e56348c7d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:37.803602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463897e3-7a8a-4a21-83c9-6e56348c7d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:37.804488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463897e3-7a8a-4a21-83c9-6e56348c7d37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:37.804630 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463897e3-7a8a-4a21-83c9-6e56348c7d37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:14:37.804940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463897e3-7a8a-4a21-83c9-6e56348c7d37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:37.806600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463897e3-7a8a-4a21-83c9-6e56348c7d37 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:37.822399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1198/2390] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:14:37.823400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22845b24-7b89-427b-b342-0b977414a18b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:37.832913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22845b24-7b89-427b-b342-0b977414a18b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:37.838239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14951b-a4e6-4189-95b7-ba116edb59e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:37.839381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14951b-a4e6-4189-95b7-ba116edb59e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:37.839761 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14951b-a4e6-4189-95b7-ba116edb59e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:37.840098 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14951b-a4e6-4189-95b7-ba116edb59e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:37.840898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14951b-a4e6-4189-95b7-ba116edb59e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:37.843665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14951b-a4e6-4189-95b7-ba116edb59e9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:37.872660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1193/2391] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:14:37 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:14:37.876811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463897e3-7a8a-4a21-83c9-6e56348c7d37 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:37.887309 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463897e3-7a8a-4a21-83c9-6e56348c7d37 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:37.925654 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:37.955433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be15669-36ae-4d09-b184-2c346265bdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:37.956265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be15669-36ae-4d09-b184-2c346265bdce admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:37.956620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be15669-36ae-4d09-b184-2c346265bdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:37.956898 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be15669-36ae-4d09-b184-2c346265bdce admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:37.957653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be15669-36ae-4d09-b184-2c346265bdce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:37.959579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be15669-36ae-4d09-b184-2c346265bdce admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:37.987886 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1199/2392] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:14:37 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:14:37.989209 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14951b-a4e6-4189-95b7-ba116edb59e9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:37.999847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14951b-a4e6-4189-95b7-ba116edb59e9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:38.004491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669ad02-2e38-454f-bcd0-3f160c08f249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:38.005166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669ad02-2e38-454f-bcd0-3f160c08f249 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:38.005459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669ad02-2e38-454f-bcd0-3f160c08f249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:38.005774 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669ad02-2e38-454f-bcd0-3f160c08f249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:38.006512 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669ad02-2e38-454f-bcd0-3f160c08f249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:38.008140 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669ad02-2e38-454f-bcd0-3f160c08f249 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:38.019801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1194/2393] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:14:38.023590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be15669-36ae-4d09-b184-2c346265bdce admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:38.033839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be15669-36ae-4d09-b184-2c346265bdce admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:38.038540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb2f0a-d5c9-4122-9744-2a8780735e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:38.039211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2f0a-d5c9-4122-9744-2a8780735e55 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:38.039493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2f0a-d5c9-4122-9744-2a8780735e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:38.039768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2f0a-d5c9-4122-9744-2a8780735e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:38.040506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb2f0a-d5c9-4122-9744-2a8780735e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:38.042160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb2f0a-d5c9-4122-9744-2a8780735e55 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:38.052624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1200/2394] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:14:38.057045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669ad02-2e38-454f-bcd0-3f160c08f249 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:38.067451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669ad02-2e38-454f-bcd0-3f160c08f249 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:38.072188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46faaf-e28e-41cf-8959-bf76d7775267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:38.073123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46faaf-e28e-41cf-8959-bf76d7775267 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:38.073348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46faaf-e28e-41cf-8959-bf76d7775267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:38.073644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46faaf-e28e-41cf-8959-bf76d7775267 admin admin] PATH_INFO: `/v3/projects/02795b95a9b84564999233883b763e4f/users/2495a8406c184164982d136edc351cfd/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:38.089378 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e46faaf-e28e-41cf-8959-bf76d7775267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02795b95a9b84564999233883b763e4f, user_id=2495a8406c184164982d136edc351cfd, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:38.092183 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e46faaf-e28e-41cf-8959-bf76d7775267 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:38.108042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1195/2395] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:14:38 2022] PUT /identity/v3/projects/02795b95a9b84564999233883b763e4f/users/2495a8406c184164982d136edc351cfd/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:14:38.113328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2057391-a89a-431b-ae7b-a47e99056972 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:38.113655 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2057391-a89a-431b-ae7b-a47e99056972 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:38.113941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2057391-a89a-431b-ae7b-a47e99056972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:38.133240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c2057391-a89a-431b-ae7b-a47e99056972 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:38.134611 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c2057391-a89a-431b-ae7b-a47e99056972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:38.194357 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1201/2396] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:38.204842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46faaf-e28e-41cf-8959-bf76d7775267 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:38.208217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0e46faaf-e28e-41cf-8959-bf76d7775267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:38.268519 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:38.281862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46faaf-e28e-41cf-8959-bf76d7775267 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:38.286139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc03971-ae3b-454d-a244-5b296afa3cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:38.286791 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc03971-ae3b-454d-a244-5b296afa3cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:38.287102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc03971-ae3b-454d-a244-5b296afa3cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:38.287424 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc03971-ae3b-454d-a244-5b296afa3cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:38.296649 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc03971-ae3b-454d-a244-5b296afa3cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:38.298134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc03971-ae3b-454d-a244-5b296afa3cc8 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:38.307610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1196/2397] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:39.022579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fd28f-a507-4acb-9a0f-1384be77f879 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:39.023115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fd28f-a507-4acb-9a0f-1384be77f879 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:39.023115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fd28f-a507-4acb-9a0f-1384be77f879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:39.053920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-175fd28f-a507-4acb-9a0f-1384be77f879 None None] MFA Rules not processed for user `2495a8406c184164982d136edc351cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:39.056026 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-175fd28f-a507-4acb-9a0f-1384be77f879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:39.127073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1202/2398] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:39.141921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc03971-ae3b-454d-a244-5b296afa3cc8 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:39.143473 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0dc03971-ae3b-454d-a244-5b296afa3cc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:39.202765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc03971-ae3b-454d-a244-5b296afa3cc8 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:39.206968 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2095c6-bbe8-4f0f-b045-5087dbe00b28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:39.207634 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2095c6-bbe8-4f0f-b045-5087dbe00b28 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:39.207922 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2095c6-bbe8-4f0f-b045-5087dbe00b28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:39.208204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2095c6-bbe8-4f0f-b045-5087dbe00b28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:39.217943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2095c6-bbe8-4f0f-b045-5087dbe00b28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:39.220535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2095c6-bbe8-4f0f-b045-5087dbe00b28 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:39.231531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1197/2399] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:40.636165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c425c-c191-4a72-98ac-2271797c1497 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:40.636728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c425c-c191-4a72-98ac-2271797c1497 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:40.636728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c425c-c191-4a72-98ac-2271797c1497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:40.655768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-342c425c-c191-4a72-98ac-2271797c1497 None None] MFA Rules not processed for user `2495a8406c184164982d136edc351cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:14:40.656982 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-342c425c-c191-4a72-98ac-2271797c1497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:40.665853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1203/2400] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:14:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:14:40.972398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2095c6-bbe8-4f0f-b045-5087dbe00b28 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:40.974073 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fb2095c6-bbe8-4f0f-b045-5087dbe00b28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:40.986965 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2095c6-bbe8-4f0f-b045-5087dbe00b28 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:40.991082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f868e-8a5a-437a-b129-0cdf706dc00f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:40.991737 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f868e-8a5a-437a-b129-0cdf706dc00f service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:40.992006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f868e-8a5a-437a-b129-0cdf706dc00f service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:40.992283 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f868e-8a5a-437a-b129-0cdf706dc00f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:40.994234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-539f868e-8a5a-437a-b129-0cdf706dc00f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:41.004559 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f868e-8a5a-437a-b129-0cdf706dc00f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:41.006050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f868e-8a5a-437a-b129-0cdf706dc00f service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:41.014218 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1198/2401] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:44.049362 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c425c-c191-4a72-98ac-2271797c1497 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:14:44.052014 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-342c425c-c191-4a72-98ac-2271797c1497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:44.104811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:44.149062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c425c-c191-4a72-98ac-2271797c1497 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:14:44.155586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e15ea-8a91-47ff-985c-9a382ccf288f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:14:44.156303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e15ea-8a91-47ff-985c-9a382ccf288f service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:14:44.156606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e15ea-8a91-47ff-985c-9a382ccf288f service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:14:44.156877 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e15ea-8a91-47ff-985c-9a382ccf288f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:14:44.159149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-499e15ea-8a91-47ff-985c-9a382ccf288f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:14:44.171723 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e15ea-8a91-47ff-985c-9a382ccf288f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:14:44.173398 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e15ea-8a91-47ff-985c-9a382ccf288f service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:14:44.184501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1204/2402] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:14:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:14:51.010650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=38 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:14:54.180097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=17 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:08.159047 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f868e-8a5a-437a-b129-0cdf706dc00f service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:08.168070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f868e-8a5a-437a-b129-0cdf706dc00f service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:08.171776 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aeb57b-22df-481e-8fef-c701f84ca0f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:08.172418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aeb57b-22df-481e-8fef-c701f84ca0f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:08.172697 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aeb57b-22df-481e-8fef-c701f84ca0f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:08.172972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aeb57b-22df-481e-8fef-c701f84ca0f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:08.174757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a7aeb57b-22df-481e-8fef-c701f84ca0f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:08.187294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aeb57b-22df-481e-8fef-c701f84ca0f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:08.188831 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aeb57b-22df-481e-8fef-c701f84ca0f6 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:08.199478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1199/2403] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:15:09.043046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e15ea-8a91-47ff-985c-9a382ccf288f service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:09.053342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e15ea-8a91-47ff-985c-9a382ccf288f service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:09.057399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81338fc7-63a8-4ae6-a3d0-cf257ad67cfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:09.058069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81338fc7-63a8-4ae6-a3d0-cf257ad67cfa service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:09.058350 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81338fc7-63a8-4ae6-a3d0-cf257ad67cfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:09.058619 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81338fc7-63a8-4ae6-a3d0-cf257ad67cfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:09.060727 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-81338fc7-63a8-4ae6-a3d0-cf257ad67cfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:09.074476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81338fc7-63a8-4ae6-a3d0-cf257ad67cfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:09.075949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81338fc7-63a8-4ae6-a3d0-cf257ad67cfa service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:09.086345 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1205/2404] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:12.909063 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aeb57b-22df-481e-8fef-c701f84ca0f6 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:12.912034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a7aeb57b-22df-481e-8fef-c701f84ca0f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:12.989238 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81338fc7-63a8-4ae6-a3d0-cf257ad67cfa service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:12.991765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-81338fc7-63a8-4ae6-a3d0-cf257ad67cfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:13.004945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81338fc7-63a8-4ae6-a3d0-cf257ad67cfa service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:13.013216 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3148faa-3316-4c9b-a2b8-3fc76d6c3db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:13.014392 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3148faa-3316-4c9b-a2b8-3fc76d6c3db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:13.015016 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3148faa-3316-4c9b-a2b8-3fc76d6c3db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:13.015296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3148faa-3316-4c9b-a2b8-3fc76d6c3db0 admin admin] PATH_INFO: `/v3/users/dfba262d439a4253bf1c2c0f114d0cdc` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:13.017071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3148faa-3316-4c9b-a2b8-3fc76d6c3db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfba262d439a4253bf1c2c0f114d0cdc)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:13.019540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3148faa-3316-4c9b-a2b8-3fc76d6c3db0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:13.020168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aeb57b-22df-481e-8fef-c701f84ca0f6 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:13.025655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2045cd-b173-4334-bb28-e3e308a76202 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:13.026335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2045cd-b173-4334-bb28-e3e308a76202 service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:13.026621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2045cd-b173-4334-bb28-e3e308a76202 service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:13.026900 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2045cd-b173-4334-bb28-e3e308a76202 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:13.028727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8a2045cd-b173-4334-bb28-e3e308a76202 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:13.042123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2045cd-b173-4334-bb28-e3e308a76202 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:13.043792 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2045cd-b173-4334-bb28-e3e308a76202 service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:13.055300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1200/2405] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:13.086709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f3148faa-3316-4c9b-a2b8-3fc76d6c3db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfba262d439a4253bf1c2c0f114d0cdc'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:13.087575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f3148faa-3316-4c9b-a2b8-3fc76d6c3db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfba262d439a4253bf1c2c0f114d0cdc'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:13.101132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f3148faa-3316-4c9b-a2b8-3fc76d6c3db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfba262d439a4253bf1c2c0f114d0cdc'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:13.114056 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f3148faa-3316-4c9b-a2b8-3fc76d6c3db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfba262d439a4253bf1c2c0f114d0cdc'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:13.114731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f3148faa-3316-4c9b-a2b8-3fc76d6c3db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfba262d439a4253bf1c2c0f114d0cdc'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:13.131268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1206/2406] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:12 2022] DELETE /identity/v3/users/dfba262d439a4253bf1c2c0f114d0cdc => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:13.400775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2045cd-b173-4334-bb28-e3e308a76202 service placement] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:13.416098 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2045cd-b173-4334-bb28-e3e308a76202 service placement] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:13.506516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:13.507486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:13.507843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:13.508193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d admin admin] PATH_INFO: `/v3/projects/2f559cbed35f4fbb8fe71906696733e7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:13.509465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f559cbed35f4fbb8fe71906696733e7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:13.511522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:13.563053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f559cbed35f4fbb8fe71906696733e7', 'request_id': 'req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:13.570106 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f559cbed35f4fbb8fe71906696733e7', 'request_id': 'req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:13.575812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f559cbed35f4fbb8fe71906696733e7', 'request_id': 'req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:13.590336 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d admin admin] The token cache is being invalidate because project 2f559cbed35f4fbb8fe71906696733e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:13.590625 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:13.591259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e2bc8793-23dd-4c32-ab49-dce0a7338e8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:13.592875 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1201/2407] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:13 2022] DELETE /identity/v3/projects/2f559cbed35f4fbb8fe71906696733e7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:13.597933 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85961a66-e077-4bbd-8eff-573c6c49044c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:13.598237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85961a66-e077-4bbd-8eff-573c6c49044c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:13.598525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85961a66-e077-4bbd-8eff-573c6c49044c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:13.616036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-85961a66-e077-4bbd-8eff-573c6c49044c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:13.617253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-85961a66-e077-4bbd-8eff-573c6c49044c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:13.714121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1207/2408] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:13.722737 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a443a8-5265-4978-b735-45454e07e171 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:13.723031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a443a8-5265-4978-b735-45454e07e171 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:13.723309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a443a8-5265-4978-b735-45454e07e171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:13.736358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:13.744546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-71a443a8-5265-4978-b735-45454e07e171 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:13.746989 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-71a443a8-5265-4978-b735-45454e07e171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:13.758056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1202/2409] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:13.762653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85961a66-e077-4bbd-8eff-573c6c49044c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:13.774363 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85961a66-e077-4bbd-8eff-573c6c49044c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:13.779265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4db452-e182-448b-8707-de9ea709b334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:13.779941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4db452-e182-448b-8707-de9ea709b334 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:13.780226 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4db452-e182-448b-8707-de9ea709b334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:13.780501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4db452-e182-448b-8707-de9ea709b334 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:13.781235 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4db452-e182-448b-8707-de9ea709b334 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:15:13.781544 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4db452-e182-448b-8707-de9ea709b334 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:13.783197 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4db452-e182-448b-8707-de9ea709b334 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:13.797007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1208/2410] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 10:15:13.801997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a443a8-5265-4978-b735-45454e07e171 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:13.812568 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a443a8-5265-4978-b735-45454e07e171 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:13.817444 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb0925-7d28-4028-976b-b9dc5d10fae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:13.818135 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0925-7d28-4028-976b-b9dc5d10fae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:13.819108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0925-7d28-4028-976b-b9dc5d10fae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:13.819398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0925-7d28-4028-976b-b9dc5d10fae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:13.820117 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb0925-7d28-4028-976b-b9dc5d10fae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:13.822132 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb0925-7d28-4028-976b-b9dc5d10fae7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:13.870399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1203/2411] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:13 2022] POST /identity/v3/projects => generated 357 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:13.874430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4db452-e182-448b-8707-de9ea709b334 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:13.889110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4db452-e182-448b-8707-de9ea709b334 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:13.929800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb0925-7d28-4028-976b-b9dc5d10fae7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:13.931979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f5fb0925-7d28-4028-976b-b9dc5d10fae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:13.969078 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:13.997021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e54aa3-72fe-4837-b3d8-7e28e69ec08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:13.998807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e54aa3-72fe-4837-b3d8-7e28e69ec08c admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:13.999371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e54aa3-72fe-4837-b3d8-7e28e69ec08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:13.999757 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e54aa3-72fe-4837-b3d8-7e28e69ec08c admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.000728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e54aa3-72fe-4837-b3d8-7e28e69ec08c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:14.003301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e54aa3-72fe-4837-b3d8-7e28e69ec08c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:14.029734 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1209/2412] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:13 2022] POST /identity/v3/users => generated 279 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:14.034374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e54aa3-72fe-4837-b3d8-7e28e69ec08c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:14.051900 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e54aa3-72fe-4837-b3d8-7e28e69ec08c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:14.058224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6cb4f-47e8-41a6-b920-c63ead1911b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:14.059299 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6cb4f-47e8-41a6-b920-c63ead1911b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.059733 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6cb4f-47e8-41a6-b920-c63ead1911b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.060387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6cb4f-47e8-41a6-b920-c63ead1911b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.061138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6cb4f-47e8-41a6-b920-c63ead1911b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:14.063943 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6cb4f-47e8-41a6-b920-c63ead1911b7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:14.066444 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb0925-7d28-4028-976b-b9dc5d10fae7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:14.072541 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217e066-5e20-410b-ac85-e4e56ed15166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:14.073540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217e066-5e20-410b-ac85-e4e56ed15166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.074651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217e066-5e20-410b-ac85-e4e56ed15166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.075027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217e066-5e20-410b-ac85-e4e56ed15166 admin admin] PATH_INFO: `/v3/users/88d2351fc3b74ef5a530fded1be6b997` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.076491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217e066-5e20-410b-ac85-e4e56ed15166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d2351fc3b74ef5a530fded1be6b997)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:14.078603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217e066-5e20-410b-ac85-e4e56ed15166 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:14.078964 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1210/2413] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:14.082461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6cb4f-47e8-41a6-b920-c63ead1911b7 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:14.101692 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6cb4f-47e8-41a6-b920-c63ead1911b7 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:14.111097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11df51b-5391-4dab-a650-54b698aac0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:14.112177 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11df51b-5391-4dab-a650-54b698aac0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.112441 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11df51b-5391-4dab-a650-54b698aac0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.112829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11df51b-5391-4dab-a650-54b698aac0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.113559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11df51b-5391-4dab-a650-54b698aac0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:14.115413 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11df51b-5391-4dab-a650-54b698aac0c4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:14.127037 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1211/2414] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:15:14.131301 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11df51b-5391-4dab-a650-54b698aac0c4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:14.145507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e217e066-5e20-410b-ac85-e4e56ed15166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d2351fc3b74ef5a530fded1be6b997'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:14.146397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e217e066-5e20-410b-ac85-e4e56ed15166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d2351fc3b74ef5a530fded1be6b997'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:14.146610 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11df51b-5391-4dab-a650-54b698aac0c4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:14.151316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a09c7c-0063-4deb-8b4b-1b8a21031c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:14.152210 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a09c7c-0063-4deb-8b4b-1b8a21031c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.152492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a09c7c-0063-4deb-8b4b-1b8a21031c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.152767 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a09c7c-0063-4deb-8b4b-1b8a21031c94 admin admin] PATH_INFO: `/v3/projects/d9e22331c94e45a7aab639eef31482c2/users/5f136228a90f4527b6c8de2bafe57a70/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.155241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e217e066-5e20-410b-ac85-e4e56ed15166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d2351fc3b74ef5a530fded1be6b997'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:14.168298 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e217e066-5e20-410b-ac85-e4e56ed15166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d2351fc3b74ef5a530fded1be6b997'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:14.168982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e217e066-5e20-410b-ac85-e4e56ed15166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d2351fc3b74ef5a530fded1be6b997'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:14.169328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a09c7c-0063-4deb-8b4b-1b8a21031c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e22331c94e45a7aab639eef31482c2, user_id=5f136228a90f4527b6c8de2bafe57a70, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:14.172039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a09c7c-0063-4deb-8b4b-1b8a21031c94 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:14.182064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1204/2415] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:13 2022] DELETE /identity/v3/users/88d2351fc3b74ef5a530fded1be6b997 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:14.189032 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1212/2416] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:14 2022] PUT /identity/v3/projects/d9e22331c94e45a7aab639eef31482c2/users/5f136228a90f4527b6c8de2bafe57a70/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:14.199545 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67324ad6-d98a-4c74-b177-0e60b1d3d193 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.199834 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67324ad6-d98a-4c74-b177-0e60b1d3d193 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.200113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67324ad6-d98a-4c74-b177-0e60b1d3d193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.226709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-67324ad6-d98a-4c74-b177-0e60b1d3d193 None None] MFA Rules not processed for user `5f136228a90f4527b6c8de2bafe57a70`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:14.227921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-67324ad6-d98a-4c74-b177-0e60b1d3d193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:14.271270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:14.310034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1205/2417] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:14 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:14.314581 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a09c7c-0063-4deb-8b4b-1b8a21031c94 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:14.317961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-23a09c7c-0063-4deb-8b4b-1b8a21031c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:14.396193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67324ad6-d98a-4c74-b177-0e60b1d3d193 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:14.399642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-67324ad6-d98a-4c74-b177-0e60b1d3d193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:14.429722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a09c7c-0063-4deb-8b4b-1b8a21031c94 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:14.435909 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee702670-36b9-403b-904d-869961c528e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:14.436844 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee702670-36b9-403b-904d-869961c528e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.437359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee702670-36b9-403b-904d-869961c528e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.437742 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee702670-36b9-403b-904d-869961c528e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.439112 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee702670-36b9-403b-904d-869961c528e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:14.441767 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee702670-36b9-403b-904d-869961c528e4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:14.485714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1213/2418] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:14 2022] POST /identity/v3/projects => generated 357 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:14.497196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee702670-36b9-403b-904d-869961c528e4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:14.550853 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67324ad6-d98a-4c74-b177-0e60b1d3d193 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:14.589242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee702670-36b9-403b-904d-869961c528e4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:14.594919 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bc1da-3c7d-4845-aa51-ecf923a56204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:14.595871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bc1da-3c7d-4845-aa51-ecf923a56204 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.596241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bc1da-3c7d-4845-aa51-ecf923a56204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.596615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bc1da-3c7d-4845-aa51-ecf923a56204 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.597416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bc1da-3c7d-4845-aa51-ecf923a56204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:14.599694 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bc1da-3c7d-4845-aa51-ecf923a56204 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:14.632288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1214/2419] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:14 2022] POST /identity/v3/users => generated 285 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:14.637631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bc1da-3c7d-4845-aa51-ecf923a56204 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:14.640192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526db80b-74fd-41a6-93ba-00853c399968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:14.642212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526db80b-74fd-41a6-93ba-00853c399968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.644948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526db80b-74fd-41a6-93ba-00853c399968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.645334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526db80b-74fd-41a6-93ba-00853c399968 admin admin] PATH_INFO: `/v3/projects/a57f93d66f534107a228d908afa64890` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.647602 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526db80b-74fd-41a6-93ba-00853c399968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a57f93d66f534107a228d908afa64890)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:14.649890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526db80b-74fd-41a6-93ba-00853c399968 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:14.652099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:14.657091 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bc1da-3c7d-4845-aa51-ecf923a56204 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:14.675751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22873474-5eb9-4e72-9f6d-0dfb2710a3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:14.676440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22873474-5eb9-4e72-9f6d-0dfb2710a3bd admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.677115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22873474-5eb9-4e72-9f6d-0dfb2710a3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.677115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22873474-5eb9-4e72-9f6d-0dfb2710a3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.677750 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22873474-5eb9-4e72-9f6d-0dfb2710a3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:14.679417 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22873474-5eb9-4e72-9f6d-0dfb2710a3bd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:14.695527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1215/2420] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:14.699727 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22873474-5eb9-4e72-9f6d-0dfb2710a3bd admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:14.717653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22873474-5eb9-4e72-9f6d-0dfb2710a3bd admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:14.728122 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-526db80b-74fd-41a6-93ba-00853c399968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a57f93d66f534107a228d908afa64890', 'request_id': 'req-526db80b-74fd-41a6-93ba-00853c399968'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:14.736805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-526db80b-74fd-41a6-93ba-00853c399968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a57f93d66f534107a228d908afa64890', 'request_id': 'req-526db80b-74fd-41a6-93ba-00853c399968'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:14.743814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-526db80b-74fd-41a6-93ba-00853c399968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a57f93d66f534107a228d908afa64890', 'request_id': 'req-526db80b-74fd-41a6-93ba-00853c399968'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:14.760406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-526db80b-74fd-41a6-93ba-00853c399968 admin admin] The token cache is being invalidate because project a57f93d66f534107a228d908afa64890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:14.760406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-526db80b-74fd-41a6-93ba-00853c399968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:14.760406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-526db80b-74fd-41a6-93ba-00853c399968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:14.762219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1206/2421] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:14 2022] DELETE /identity/v3/projects/a57f93d66f534107a228d908afa64890 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:14.767426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9f039-9433-49d4-a673-0a9a69b3d124 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.767795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9f039-9433-49d4-a673-0a9a69b3d124 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.768288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9f039-9433-49d4-a673-0a9a69b3d124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.785263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc78179-89df-413b-a460-93bb72eb8877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:14.786404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc78179-89df-413b-a460-93bb72eb8877 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.786647 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc78179-89df-413b-a460-93bb72eb8877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.787033 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc78179-89df-413b-a460-93bb72eb8877 admin admin] PATH_INFO: `/v3/projects/bd9f6859854d42a8987c2c203ecfe2f4/users/ae6f36ef60de4ff7b964e464dc675f3f/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.789939 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b5e9f039-9433-49d4-a673-0a9a69b3d124 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:14.791162 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b5e9f039-9433-49d4-a673-0a9a69b3d124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:14.801936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1207/2422] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:14.809196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc78179-89df-413b-a460-93bb72eb8877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd9f6859854d42a8987c2c203ecfe2f4, user_id=ae6f36ef60de4ff7b964e464dc675f3f, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:14.811066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3ca9c-1745-4e9e-97d7-9b722d91c445 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.811359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3ca9c-1745-4e9e-97d7-9b722d91c445 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.811639 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3ca9c-1745-4e9e-97d7-9b722d91c445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.813002 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc78179-89df-413b-a460-93bb72eb8877 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:14.836455 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1216/2423] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:14 2022] PUT /identity/v3/projects/bd9f6859854d42a8987c2c203ecfe2f4/users/ae6f36ef60de4ff7b964e464dc675f3f/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:14.837582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc78179-89df-413b-a460-93bb72eb8877 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:14.840126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-bfc78179-89df-413b-a460-93bb72eb8877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:14.846349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-90f3ca9c-1745-4e9e-97d7-9b722d91c445 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:14.847918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-90f3ca9c-1745-4e9e-97d7-9b722d91c445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:14.855951 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:14.907772 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc78179-89df-413b-a460-93bb72eb8877 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:14.912880 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981f5d0-3b1c-42f2-b10e-c7ba7e2fb2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:14.913702 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981f5d0-3b1c-42f2-b10e-c7ba7e2fb2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.913988 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981f5d0-3b1c-42f2-b10e-c7ba7e2fb2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.914263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981f5d0-3b1c-42f2-b10e-c7ba7e2fb2f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.914989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981f5d0-3b1c-42f2-b10e-c7ba7e2fb2f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:14.915466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1208/2424] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:14.916646 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981f5d0-3b1c-42f2-b10e-c7ba7e2fb2f5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:14.919360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3ca9c-1745-4e9e-97d7-9b722d91c445 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:14.926817 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1217/2425] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:14.928884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3ca9c-1745-4e9e-97d7-9b722d91c445 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:14.931109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981f5d0-3b1c-42f2-b10e-c7ba7e2fb2f5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:14.933709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7c99f-3ec7-41cf-99ac-4cb7a908be6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:14.934493 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7c99f-3ec7-41cf-99ac-4cb7a908be6c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.934848 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7c99f-3ec7-41cf-99ac-4cb7a908be6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.935210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7c99f-3ec7-41cf-99ac-4cb7a908be6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.936212 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7c99f-3ec7-41cf-99ac-4cb7a908be6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:15:14.936594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7c99f-3ec7-41cf-99ac-4cb7a908be6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:14.938925 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7c99f-3ec7-41cf-99ac-4cb7a908be6c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:14.940642 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981f5d0-3b1c-42f2-b10e-c7ba7e2fb2f5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:14.945699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da7b90-dd59-40f1-be12-1641cb446f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:14.946620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da7b90-dd59-40f1-be12-1641cb446f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.946873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da7b90-dd59-40f1-be12-1641cb446f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.947172 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da7b90-dd59-40f1-be12-1641cb446f1a admin admin] PATH_INFO: `/v3/projects/bd9f6859854d42a8987c2c203ecfe2f4/users/ae6f36ef60de4ff7b964e464dc675f3f/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:14.949234 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da7b90-dd59-40f1-be12-1641cb446f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd9f6859854d42a8987c2c203ecfe2f4, user_id=ae6f36ef60de4ff7b964e464dc675f3f, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:14.953658 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1209/2426] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 10:15:14.953796 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da7b90-dd59-40f1-be12-1641cb446f1a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:14.956396 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7c99f-3ec7-41cf-99ac-4cb7a908be6c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:14.963112 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1218/2427] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:14 2022] PUT /identity/v3/projects/bd9f6859854d42a8987c2c203ecfe2f4/users/ae6f36ef60de4ff7b964e464dc675f3f/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:14.967194 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7c99f-3ec7-41cf-99ac-4cb7a908be6c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:14.973690 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b52a-0924-4e55-8629-ed4778c7ea91 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:14.973993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b52a-0924-4e55-8629-ed4778c7ea91 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:14.974305 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b52a-0924-4e55-8629-ed4778c7ea91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:15.001115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2a72b52a-0924-4e55-8629-ed4778c7ea91 None None] MFA Rules not processed for user `ae6f36ef60de4ff7b964e464dc675f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:15.002364 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2a72b52a-0924-4e55-8629-ed4778c7ea91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:15.025027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9aeeb8-4999-40fc-8654-5cf2520babf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:15.025727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9aeeb8-4999-40fc-8654-5cf2520babf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:15.026015 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9aeeb8-4999-40fc-8654-5cf2520babf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:15.026292 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9aeeb8-4999-40fc-8654-5cf2520babf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:15.026982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9aeeb8-4999-40fc-8654-5cf2520babf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:15.028898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9aeeb8-4999-40fc-8654-5cf2520babf7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:15.049242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:15.056682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1210/2428] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:14 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:15.060881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9aeeb8-4999-40fc-8654-5cf2520babf7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:15.073454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1219/2429] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:14 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:15.085397 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b52a-0924-4e55-8629-ed4778c7ea91 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:15.086803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2a72b52a-0924-4e55-8629-ed4778c7ea91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:15.128386 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9aeeb8-4999-40fc-8654-5cf2520babf7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:15.134416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adda732-f1c4-4d3a-bf56-f252cd821c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:15.135225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adda732-f1c4-4d3a-bf56-f252cd821c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:15.135458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adda732-f1c4-4d3a-bf56-f252cd821c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:15.135724 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adda732-f1c4-4d3a-bf56-f252cd821c28 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:15.136934 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adda732-f1c4-4d3a-bf56-f252cd821c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:15.138585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adda732-f1c4-4d3a-bf56-f252cd821c28 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:15.146706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b52a-0924-4e55-8629-ed4778c7ea91 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:15.163087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1211/2430] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:15 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:15.167439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adda732-f1c4-4d3a-bf56-f252cd821c28 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:15.177124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adda732-f1c4-4d3a-bf56-f252cd821c28 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:15.181933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af305e95-981d-4c8f-912c-a559c455c9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:15.182670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af305e95-981d-4c8f-912c-a559c455c9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:15.182957 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af305e95-981d-4c8f-912c-a559c455c9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:15.183236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af305e95-981d-4c8f-912c-a559c455c9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:15.183955 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af305e95-981d-4c8f-912c-a559c455c9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:15.185553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af305e95-981d-4c8f-912c-a559c455c9c1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:15.190837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:15.196553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1212/2431] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:15.199971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce857b-ae35-4ab2-b85b-feafbfbfbdd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:15.200296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce857b-ae35-4ab2-b85b-feafbfbfbdd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:15.200619 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce857b-ae35-4ab2-b85b-feafbfbfbdd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:15.200898 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce857b-ae35-4ab2-b85b-feafbfbfbdd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:15.201330 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af305e95-981d-4c8f-912c-a559c455c9c1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:15.211029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af305e95-981d-4c8f-912c-a559c455c9c1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:15.216161 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccde26f-00fe-4da8-8fe5-d3dc7ad322fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:15.216807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccde26f-00fe-4da8-8fe5-d3dc7ad322fa admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:15.217093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccde26f-00fe-4da8-8fe5-d3dc7ad322fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:15.217376 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccde26f-00fe-4da8-8fe5-d3dc7ad322fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:15.218192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccde26f-00fe-4da8-8fe5-d3dc7ad322fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:15.219790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccde26f-00fe-4da8-8fe5-d3dc7ad322fa admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:15.230724 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1213/2432] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:15:15.234987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccde26f-00fe-4da8-8fe5-d3dc7ad322fa admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:15.245338 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccde26f-00fe-4da8-8fe5-d3dc7ad322fa admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:15.250880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd691e7-ae08-4421-b5d9-473e78ef1828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:15.251754 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd691e7-ae08-4421-b5d9-473e78ef1828 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:15.252065 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd691e7-ae08-4421-b5d9-473e78ef1828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:15.252343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd691e7-ae08-4421-b5d9-473e78ef1828 admin admin] PATH_INFO: `/v3/projects/e1ae2bf7ed184f7a99c784aececd1d15/users/044f8d0a53dd435480778bd8bf291b50/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:15.263303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce857b-ae35-4ab2-b85b-feafbfbfbdd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:15.264788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce857b-ae35-4ab2-b85b-feafbfbfbdd5 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:15.267917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd691e7-ae08-4421-b5d9-473e78ef1828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ae2bf7ed184f7a99c784aececd1d15, user_id=044f8d0a53dd435480778bd8bf291b50, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:15.270628 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd691e7-ae08-4421-b5d9-473e78ef1828 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:15.274579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1220/2433] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:15 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:15:15.285553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1214/2434] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:15 2022] PUT /identity/v3/projects/e1ae2bf7ed184f7a99c784aececd1d15/users/044f8d0a53dd435480778bd8bf291b50/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:15.297672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5468f38-23e6-46c9-897d-4c6ac9143722 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:15.297954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5468f38-23e6-46c9-897d-4c6ac9143722 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:15.298347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5468f38-23e6-46c9-897d-4c6ac9143722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:15.328993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d5468f38-23e6-46c9-897d-4c6ac9143722 None None] MFA Rules not processed for user `044f8d0a53dd435480778bd8bf291b50`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:15.330249 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d5468f38-23e6-46c9-897d-4c6ac9143722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:15.349056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd691e7-ae08-4421-b5d9-473e78ef1828 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:15.372915 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:15.398644 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1221/2435] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:15.405303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd691e7-ae08-4421-b5d9-473e78ef1828 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:15.409375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3f771-845a-44d6-881b-7de60f30b8d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:15.410059 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5468f38-23e6-46c9-897d-4c6ac9143722 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:15.410201 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3f771-845a-44d6-881b-7de60f30b8d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:15.410341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3f771-845a-44d6-881b-7de60f30b8d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:15.410622 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3f771-845a-44d6-881b-7de60f30b8d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:15.411554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d5468f38-23e6-46c9-897d-4c6ac9143722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:15.412835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ccb3f771-845a-44d6-881b-7de60f30b8d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:15.477151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3f771-845a-44d6-881b-7de60f30b8d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:15.478647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3f771-845a-44d6-881b-7de60f30b8d2 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:15.485827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5468f38-23e6-46c9-897d-4c6ac9143722 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:15.488135 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1215/2436] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:15 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:15:15.491397 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37225203-fada-4116-88c7-8f5d1b3c8718 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:15.492052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37225203-fada-4116-88c7-8f5d1b3c8718 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:15.492332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37225203-fada-4116-88c7-8f5d1b3c8718 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:15.492605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37225203-fada-4116-88c7-8f5d1b3c8718 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:15.500984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37225203-fada-4116-88c7-8f5d1b3c8718 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:15.502624 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37225203-fada-4116-88c7-8f5d1b3c8718 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:15.511619 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1222/2437] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:15:25.483506 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=63,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:25.507611 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=43 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:30.350631 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3f771-845a-44d6-881b-7de60f30b8d2 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:30.419190 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3f771-845a-44d6-881b-7de60f30b8d2 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:30.424799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:30.425678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:30.425991 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:30.426257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] PATH_INFO: `/v3/users/044f8d0a53dd435480778bd8bf291b50` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:30.427528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=044f8d0a53dd435480778bd8bf291b50)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:30.429461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:30.483978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '044f8d0a53dd435480778bd8bf291b50'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:30.484615 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '044f8d0a53dd435480778bd8bf291b50'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:30.493396 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '044f8d0a53dd435480778bd8bf291b50'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:30.510081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '044f8d0a53dd435480778bd8bf291b50'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:30.510985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '044f8d0a53dd435480778bd8bf291b50'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:30.524242 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1216/2438] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:30 2022] DELETE /identity/v3/users/044f8d0a53dd435480778bd8bf291b50 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:30.528807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5075b50-3645-4faa-aa64-2ff9b4872db8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:30.529299 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5075b50-3645-4faa-aa64-2ff9b4872db8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:30.529498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5075b50-3645-4faa-aa64-2ff9b4872db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:30.555075 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b5075b50-3645-4faa-aa64-2ff9b4872db8 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:30.556323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b5075b50-3645-4faa-aa64-2ff9b4872db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:30.583528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:30.585030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:30.618205 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1223/2439] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:30.628458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5075b50-3645-4faa-aa64-2ff9b4872db8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:30.630059 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b5075b50-3645-4faa-aa64-2ff9b4872db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:30.646167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0b3a2-8f2e-4504-acc0-209b411f34dd admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:30.650052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a1452-0f0b-4c88-84be-732d22471022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:30.650696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a1452-0f0b-4c88-84be-732d22471022 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:30.650984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a1452-0f0b-4c88-84be-732d22471022 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:30.651256 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a1452-0f0b-4c88-84be-732d22471022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:30.653397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f56a1452-0f0b-4c88-84be-732d22471022 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:30.671805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a1452-0f0b-4c88-84be-732d22471022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:30.673251 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a1452-0f0b-4c88-84be-732d22471022 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:30.684190 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1217/2440] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:15:30.696655 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5075b50-3645-4faa-aa64-2ff9b4872db8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:30.700255 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66190be-892b-44f8-a974-dd81f302a60d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:30.700975 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66190be-892b-44f8-a974-dd81f302a60d service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:30.701325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66190be-892b-44f8-a974-dd81f302a60d service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:30.701720 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66190be-892b-44f8-a974-dd81f302a60d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:30.711231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66190be-892b-44f8-a974-dd81f302a60d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:30.712772 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66190be-892b-44f8-a974-dd81f302a60d service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:30.723745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1224/2441] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:31.440866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a1452-0f0b-4c88-84be-732d22471022 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:31.442514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f56a1452-0f0b-4c88-84be-732d22471022 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:31.455193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a1452-0f0b-4c88-84be-732d22471022 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:31.459986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:31.460947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:31.461304 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:31.461699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43 admin admin] PATH_INFO: `/v3/projects/e1ae2bf7ed184f7a99c784aececd1d15` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:31.463329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1ae2bf7ed184f7a99c784aececd1d15)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:31.465333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:31.484173 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:31.519708 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1ae2bf7ed184f7a99c784aececd1d15', 'request_id': 'req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:31.526048 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1ae2bf7ed184f7a99c784aececd1d15', 'request_id': 'req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:31.531933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1ae2bf7ed184f7a99c784aececd1d15', 'request_id': 'req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:31.545324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43 admin admin] The token cache is being invalidate because project e1ae2bf7ed184f7a99c784aececd1d15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:31.545627 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:31.546905 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8b6eaaca-8311-4995-b911-c1dcbf3a8d43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:31.548873 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1218/2442] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:31 2022] DELETE /identity/v3/projects/e1ae2bf7ed184f7a99c784aececd1d15 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:31.553575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69082ac-7952-4800-84ad-22e63f8240d5 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:31.554099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69082ac-7952-4800-84ad-22e63f8240d5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:31.554203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69082ac-7952-4800-84ad-22e63f8240d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:31.572802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b69082ac-7952-4800-84ad-22e63f8240d5 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:31.574046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b69082ac-7952-4800-84ad-22e63f8240d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:31.639900 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1225/2443] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:31.647906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9065d20-3e18-46d1-a937-8e71d4b9a1ad None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:31.648271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9065d20-3e18-46d1-a937-8e71d4b9a1ad None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:31.648620 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9065d20-3e18-46d1-a937-8e71d4b9a1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:31.667211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e9065d20-3e18-46d1-a937-8e71d4b9a1ad None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:31.668421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e9065d20-3e18-46d1-a937-8e71d4b9a1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:31.677370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1219/2444] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:31.681414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69082ac-7952-4800-84ad-22e63f8240d5 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:31.691632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69082ac-7952-4800-84ad-22e63f8240d5 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:31.696220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac1b86-f2a2-4efc-98c5-aea841567380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:31.696884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac1b86-f2a2-4efc-98c5-aea841567380 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:31.697168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac1b86-f2a2-4efc-98c5-aea841567380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:31.697445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac1b86-f2a2-4efc-98c5-aea841567380 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:31.698211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac1b86-f2a2-4efc-98c5-aea841567380 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:15:31.698519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac1b86-f2a2-4efc-98c5-aea841567380 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:31.700132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac1b86-f2a2-4efc-98c5-aea841567380 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:31.712257 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1226/2445] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:15:31.717051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9065d20-3e18-46d1-a937-8e71d4b9a1ad None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:31.726550 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9065d20-3e18-46d1-a937-8e71d4b9a1ad None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:31.731847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c1bdb-72d3-416e-8b62-e0a0d21679ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:31.732510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c1bdb-72d3-416e-8b62-e0a0d21679ef admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:31.732789 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c1bdb-72d3-416e-8b62-e0a0d21679ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:31.733070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c1bdb-72d3-416e-8b62-e0a0d21679ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:31.733879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c1bdb-72d3-416e-8b62-e0a0d21679ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:31.735857 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c1bdb-72d3-416e-8b62-e0a0d21679ef admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:31.762995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1220/2446] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:31 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:31.767146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac1b86-f2a2-4efc-98c5-aea841567380 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:31.777444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac1b86-f2a2-4efc-98c5-aea841567380 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:31.808022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:31.839063 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5bd2f-a62d-452d-96a3-b41018d519f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:31.839757 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5bd2f-a62d-452d-96a3-b41018d519f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:31.840037 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5bd2f-a62d-452d-96a3-b41018d519f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:31.840315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5bd2f-a62d-452d-96a3-b41018d519f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:31.841048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5bd2f-a62d-452d-96a3-b41018d519f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:31.843087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5bd2f-a62d-452d-96a3-b41018d519f4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:31.867029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1227/2447] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:31 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:31.871457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c1bdb-72d3-416e-8b62-e0a0d21679ef admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:31.882895 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c1bdb-72d3-416e-8b62-e0a0d21679ef admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:31.888308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ca438-4804-49b6-9a34-4d6baca95adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:31.888975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ca438-4804-49b6-9a34-4d6baca95adb admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:31.889266 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ca438-4804-49b6-9a34-4d6baca95adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:31.889541 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ca438-4804-49b6-9a34-4d6baca95adb admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:31.891001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ca438-4804-49b6-9a34-4d6baca95adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:31.892634 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ca438-4804-49b6-9a34-4d6baca95adb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:31.907864 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1221/2448] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:15:31.909376 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5bd2f-a62d-452d-96a3-b41018d519f4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:31.920388 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5bd2f-a62d-452d-96a3-b41018d519f4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:31.926325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bae33e-0b31-42e0-8d68-1fb671f6ca9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:31.926995 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bae33e-0b31-42e0-8d68-1fb671f6ca9c admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:31.927298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bae33e-0b31-42e0-8d68-1fb671f6ca9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:31.927555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bae33e-0b31-42e0-8d68-1fb671f6ca9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:31.928279 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bae33e-0b31-42e0-8d68-1fb671f6ca9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:31.929946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bae33e-0b31-42e0-8d68-1fb671f6ca9c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:31.954470 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1228/2449] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:31.957750 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ca438-4804-49b6-9a34-4d6baca95adb admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:31.969594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ca438-4804-49b6-9a34-4d6baca95adb admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:31.974641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4211655-70cd-44fd-b659-ef1a0c95e7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:31.975696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4211655-70cd-44fd-b659-ef1a0c95e7b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:31.976058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4211655-70cd-44fd-b659-ef1a0c95e7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:31.976411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4211655-70cd-44fd-b659-ef1a0c95e7b1 admin admin] PATH_INFO: `/v3/projects/4d25d19560bc4691ae77d8df9bb8ae63/users/97aa396db9d14346bcd54298c6518d34/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:31.998588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4211655-70cd-44fd-b659-ef1a0c95e7b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d25d19560bc4691ae77d8df9bb8ae63, user_id=97aa396db9d14346bcd54298c6518d34, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:32.004730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4211655-70cd-44fd-b659-ef1a0c95e7b1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:32.035189 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1222/2450] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:31 2022] PUT /identity/v3/projects/4d25d19560bc4691ae77d8df9bb8ae63/users/97aa396db9d14346bcd54298c6518d34/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:32.047822 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef762e3-ee0a-4b92-90c7-03fea353ea44 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:32.048534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef762e3-ee0a-4b92-90c7-03fea353ea44 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:32.048891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef762e3-ee0a-4b92-90c7-03fea353ea44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:32.082954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7ef762e3-ee0a-4b92-90c7-03fea353ea44 None None] MFA Rules not processed for user `97aa396db9d14346bcd54298c6518d34`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:32.084786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7ef762e3-ee0a-4b92-90c7-03fea353ea44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:32.200055 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1229/2451] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:32 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:32.201494 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4211655-70cd-44fd-b659-ef1a0c95e7b1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:32.305968 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4211655-70cd-44fd-b659-ef1a0c95e7b1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:32.319140 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360404ca-5954-4603-84b7-54898e0f1f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:32.319837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360404ca-5954-4603-84b7-54898e0f1f4e admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:32.320121 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360404ca-5954-4603-84b7-54898e0f1f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:32.320398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360404ca-5954-4603-84b7-54898e0f1f4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:32.321125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360404ca-5954-4603-84b7-54898e0f1f4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:32.323219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360404ca-5954-4603-84b7-54898e0f1f4e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:32.378508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1223/2452] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:32 2022] POST /identity/v3/projects => generated 365 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:32.384122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef762e3-ee0a-4b92-90c7-03fea353ea44 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:32.404889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef762e3-ee0a-4b92-90c7-03fea353ea44 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:32.487127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c18351-564e-4e5c-a0cf-56c04eed841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:32.487127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c18351-564e-4e5c-a0cf-56c04eed841d admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:32.487474 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c18351-564e-4e5c-a0cf-56c04eed841d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:32.487660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c18351-564e-4e5c-a0cf-56c04eed841d admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:32.488486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c18351-564e-4e5c-a0cf-56c04eed841d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:32.490553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c18351-564e-4e5c-a0cf-56c04eed841d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:32.521662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1230/2453] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:32 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:32.525152 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360404ca-5954-4603-84b7-54898e0f1f4e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:32.536432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360404ca-5954-4603-84b7-54898e0f1f4e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:32.542308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8b9dc-a69b-42c6-960b-68cf7b0c892c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:32.543334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8b9dc-a69b-42c6-960b-68cf7b0c892c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:32.543807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8b9dc-a69b-42c6-960b-68cf7b0c892c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:32.544090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8b9dc-a69b-42c6-960b-68cf7b0c892c admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:32.544836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8b9dc-a69b-42c6-960b-68cf7b0c892c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:32.546903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8b9dc-a69b-42c6-960b-68cf7b0c892c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:32.565528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1224/2454] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:32.569849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c18351-564e-4e5c-a0cf-56c04eed841d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:32.582138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c18351-564e-4e5c-a0cf-56c04eed841d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:32.588906 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b505b98-55d1-437a-8fc1-050691fe0e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:32.590148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b505b98-55d1-437a-8fc1-050691fe0e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:32.590594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b505b98-55d1-437a-8fc1-050691fe0e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:32.592814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b505b98-55d1-437a-8fc1-050691fe0e90 admin admin] PATH_INFO: `/v3/projects/14cb66bbbbc84f5b9c9a7b467dc7b06e/users/972b3a4e2273428eba988119ef4332b1/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:32.612397 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b505b98-55d1-437a-8fc1-050691fe0e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14cb66bbbbc84f5b9c9a7b467dc7b06e, user_id=972b3a4e2273428eba988119ef4332b1, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:32.617740 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b505b98-55d1-437a-8fc1-050691fe0e90 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:32.643537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1231/2455] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:32 2022] PUT /identity/v3/projects/14cb66bbbbc84f5b9c9a7b467dc7b06e/users/972b3a4e2273428eba988119ef4332b1/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:32.647384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8b9dc-a69b-42c6-960b-68cf7b0c892c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:32.668917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8b9dc-a69b-42c6-960b-68cf7b0c892c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:32.688470 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:32.756753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0578c3-4c43-49c5-b4ad-dae754bc3eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:32.757546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0578c3-4c43-49c5-b4ad-dae754bc3eeb admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:32.757982 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0578c3-4c43-49c5-b4ad-dae754bc3eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:32.758411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0578c3-4c43-49c5-b4ad-dae754bc3eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:32.759305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0578c3-4c43-49c5-b4ad-dae754bc3eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:32.761369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0578c3-4c43-49c5-b4ad-dae754bc3eeb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:32.777350 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1225/2456] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:32.778330 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b505b98-55d1-437a-8fc1-050691fe0e90 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:32.786291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:32.792152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b505b98-55d1-437a-8fc1-050691fe0e90 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:32.798896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49296a71-7a36-4bc7-a9b7-ce5cb9a664e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:32.800217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49296a71-7a36-4bc7-a9b7-ce5cb9a664e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:32.800551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49296a71-7a36-4bc7-a9b7-ce5cb9a664e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:32.801000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49296a71-7a36-4bc7-a9b7-ce5cb9a664e5 admin admin] PATH_INFO: `/v3/projects/14cb66bbbbc84f5b9c9a7b467dc7b06e/users/972b3a4e2273428eba988119ef4332b1/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:32.803089 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49296a71-7a36-4bc7-a9b7-ce5cb9a664e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14cb66bbbbc84f5b9c9a7b467dc7b06e, user_id=972b3a4e2273428eba988119ef4332b1, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:32.806083 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49296a71-7a36-4bc7-a9b7-ce5cb9a664e5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:32.818392 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1232/2457] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:32 2022] PUT /identity/v3/projects/14cb66bbbbc84f5b9c9a7b467dc7b06e/users/972b3a4e2273428eba988119ef4332b1/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:32.830909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a751ecc-36ae-4fbb-8cbc-61724415d34b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:32.831194 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a751ecc-36ae-4fbb-8cbc-61724415d34b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:32.831475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a751ecc-36ae-4fbb-8cbc-61724415d34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:32.863006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1a751ecc-36ae-4fbb-8cbc-61724415d34b None None] MFA Rules not processed for user `972b3a4e2273428eba988119ef4332b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:32.864246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1a751ecc-36ae-4fbb-8cbc-61724415d34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:32.967484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1226/2458] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:32.980339 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49296a71-7a36-4bc7-a9b7-ce5cb9a664e5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:32.983437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-49296a71-7a36-4bc7-a9b7-ce5cb9a664e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:33.051342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49296a71-7a36-4bc7-a9b7-ce5cb9a664e5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:33.056142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a775a8c-0da3-4ba1-b1c9-48f04f24077d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:33.057019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a775a8c-0da3-4ba1-b1c9-48f04f24077d service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:33.057303 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a775a8c-0da3-4ba1-b1c9-48f04f24077d service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:33.057573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a775a8c-0da3-4ba1-b1c9-48f04f24077d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:33.069088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a775a8c-0da3-4ba1-b1c9-48f04f24077d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:33.070683 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a775a8c-0da3-4ba1-b1c9-48f04f24077d service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:33.081336 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1233/2459] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:33.589025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a751ecc-36ae-4fbb-8cbc-61724415d34b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:33.656051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a751ecc-36ae-4fbb-8cbc-61724415d34b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:33.661192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:33.662578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:33.662953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:33.663280 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] PATH_INFO: `/v3/users/97aa396db9d14346bcd54298c6518d34` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:33.666011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97aa396db9d14346bcd54298c6518d34)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:33.668259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:33.736450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:33.761003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97aa396db9d14346bcd54298c6518d34'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:33.765588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97aa396db9d14346bcd54298c6518d34'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:33.782513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97aa396db9d14346bcd54298c6518d34'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:33.804054 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97aa396db9d14346bcd54298c6518d34'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:33.804832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97aa396db9d14346bcd54298c6518d34'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:33.819822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1227/2460] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:33 2022] DELETE /identity/v3/users/97aa396db9d14346bcd54298c6518d34 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:33.824639 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5322e-7d3f-4b20-a0fd-f86e80a635b1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:33.825171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5322e-7d3f-4b20-a0fd-f86e80a635b1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:33.825256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5322e-7d3f-4b20-a0fd-f86e80a635b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:33.844926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-adf5322e-7d3f-4b20-a0fd-f86e80a635b1 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:33.846318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-adf5322e-7d3f-4b20-a0fd-f86e80a635b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:33.919979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1234/2461] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:33.935496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:33.937675 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:34.050514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca8a3c-1a14-4e53-925d-d39535df46e5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:34.055090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8690a3-3fd1-4536-90cf-99ba1083ed2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:34.055954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8690a3-3fd1-4536-90cf-99ba1083ed2c service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:34.056302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8690a3-3fd1-4536-90cf-99ba1083ed2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:34.056859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8690a3-3fd1-4536-90cf-99ba1083ed2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:34.073827 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8690a3-3fd1-4536-90cf-99ba1083ed2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:34.075391 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8690a3-3fd1-4536-90cf-99ba1083ed2c service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:34.090966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1228/2462] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:34.510647 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5322e-7d3f-4b20-a0fd-f86e80a635b1 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:34.512312 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-adf5322e-7d3f-4b20-a0fd-f86e80a635b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:34.533385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5322e-7d3f-4b20-a0fd-f86e80a635b1 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:34.546254 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:34.547102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:34.547387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:34.547665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] PATH_INFO: `/v3/projects/4d25d19560bc4691ae77d8df9bb8ae63` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:34.548862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d25d19560bc4691ae77d8df9bb8ae63)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:34.550830 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:34.610105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d25d19560bc4691ae77d8df9bb8ae63', 'request_id': 'req-0b838266-defa-4e70-85cb-b0d90fd30890'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.618789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d25d19560bc4691ae77d8df9bb8ae63', 'request_id': 'req-0b838266-defa-4e70-85cb-b0d90fd30890'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.629158 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d25d19560bc4691ae77d8df9bb8ae63', 'request_id': 'req-0b838266-defa-4e70-85cb-b0d90fd30890'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.657509 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] The token cache is being invalidate because project 4d25d19560bc4691ae77d8df9bb8ae63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:34.657878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.658885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.663013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1235/2463] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:34 2022] DELETE /identity/v3/projects/4d25d19560bc4691ae77d8df9bb8ae63 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:34.667585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8690a3-3fd1-4536-90cf-99ba1083ed2c service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:34.678028 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8690a3-3fd1-4536-90cf-99ba1083ed2c service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:34.737290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:34.738231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:34.738512 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:34.738786 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] PATH_INFO: `/v3/users/972b3a4e2273428eba988119ef4332b1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:34.740263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=972b3a4e2273428eba988119ef4332b1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:34.742265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:34.797651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '972b3a4e2273428eba988119ef4332b1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.800849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '972b3a4e2273428eba988119ef4332b1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.810881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '972b3a4e2273428eba988119ef4332b1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.818285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:34.819746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:34.823027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:34.837061 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b838266-defa-4e70-85cb-b0d90fd30890 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:34.844370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '972b3a4e2273428eba988119ef4332b1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.844994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:34.845609 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '972b3a4e2273428eba988119ef4332b1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.846053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:34.846425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:34.846795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] PATH_INFO: `/v3/users/1c05b9b08df0437f9b0abb167d17aa56` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:34.848378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c05b9b08df0437f9b0abb167d17aa56)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:34.850848 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:34.868986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:34.869371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1229/2464] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:34 2022] DELETE /identity/v3/users/972b3a4e2273428eba988119ef4332b1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:34.908493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c05b9b08df0437f9b0abb167d17aa56'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.909263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c05b9b08df0437f9b0abb167d17aa56'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.926902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c05b9b08df0437f9b0abb167d17aa56'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.943811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c05b9b08df0437f9b0abb167d17aa56'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.944568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c05b9b08df0437f9b0abb167d17aa56'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:34.959904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1236/2465] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:34 2022] DELETE /identity/v3/users/1c05b9b08df0437f9b0abb167d17aa56 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:35.175302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:35.176935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:35.252664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ff94c-0df4-47e1-bf50-79ad2f5ad1c3 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:35.281103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:35.283957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:35.333869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42773d-54eb-4f28-8bce-aec0bb3ba322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:35.335198 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42773d-54eb-4f28-8bce-aec0bb3ba322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.335490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42773d-54eb-4f28-8bce-aec0bb3ba322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.335830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42773d-54eb-4f28-8bce-aec0bb3ba322 admin admin] PATH_INFO: `/v3/projects/b852926ec18f4dc695da5ec1c5582f48` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.337014 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42773d-54eb-4f28-8bce-aec0bb3ba322 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b852926ec18f4dc695da5ec1c5582f48)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:35.339384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42773d-54eb-4f28-8bce-aec0bb3ba322 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:35.360196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b87a5c-09e5-4915-954a-379f8ba20688 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:35.364775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42940b-ad84-42b6-b258-41048e484ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:35.365670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42940b-ad84-42b6-b258-41048e484ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.365953 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42940b-ad84-42b6-b258-41048e484ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.366225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42940b-ad84-42b6-b258-41048e484ee0 admin admin] PATH_INFO: `/v3/projects/14cb66bbbbc84f5b9c9a7b467dc7b06e` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.367386 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42940b-ad84-42b6-b258-41048e484ee0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14cb66bbbbc84f5b9c9a7b467dc7b06e)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:35.369299 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42940b-ad84-42b6-b258-41048e484ee0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:35.392983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4a42773d-54eb-4f28-8bce-aec0bb3ba322 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b852926ec18f4dc695da5ec1c5582f48', 'request_id': 'req-4a42773d-54eb-4f28-8bce-aec0bb3ba322'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:35.401205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4a42773d-54eb-4f28-8bce-aec0bb3ba322 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b852926ec18f4dc695da5ec1c5582f48', 'request_id': 'req-4a42773d-54eb-4f28-8bce-aec0bb3ba322'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:35.409341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4a42773d-54eb-4f28-8bce-aec0bb3ba322 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b852926ec18f4dc695da5ec1c5582f48', 'request_id': 'req-4a42773d-54eb-4f28-8bce-aec0bb3ba322'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:35.422979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4a42773d-54eb-4f28-8bce-aec0bb3ba322 admin admin] The token cache is being invalidate because project b852926ec18f4dc695da5ec1c5582f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:35.423298 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4a42773d-54eb-4f28-8bce-aec0bb3ba322 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:35.424004 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4a42773d-54eb-4f28-8bce-aec0bb3ba322 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:35.425841 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1230/2466] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:35 2022] DELETE /identity/v3/projects/b852926ec18f4dc695da5ec1c5582f48 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:35.433594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3584cc7-7757-4995-b337-b5449f17dfe1 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.433594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3584cc7-7757-4995-b337-b5449f17dfe1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.433974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3a42940b-ad84-42b6-b258-41048e484ee0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14cb66bbbbc84f5b9c9a7b467dc7b06e', 'request_id': 'req-3a42940b-ad84-42b6-b258-41048e484ee0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:35.434096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3584cc7-7757-4995-b337-b5449f17dfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.439716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3a42940b-ad84-42b6-b258-41048e484ee0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14cb66bbbbc84f5b9c9a7b467dc7b06e', 'request_id': 'req-3a42940b-ad84-42b6-b258-41048e484ee0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:35.447021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3a42940b-ad84-42b6-b258-41048e484ee0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14cb66bbbbc84f5b9c9a7b467dc7b06e', 'request_id': 'req-3a42940b-ad84-42b6-b258-41048e484ee0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:35.450727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a3584cc7-7757-4995-b337-b5449f17dfe1 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:35.452146 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a3584cc7-7757-4995-b337-b5449f17dfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:35.466264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:35.474710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3a42940b-ad84-42b6-b258-41048e484ee0 admin admin] The token cache is being invalidate because project 14cb66bbbbc84f5b9c9a7b467dc7b06e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:35.475137 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3a42940b-ad84-42b6-b258-41048e484ee0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:35.475902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3a42940b-ad84-42b6-b258-41048e484ee0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:35.477808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1237/2467] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:35 2022] DELETE /identity/v3/projects/14cb66bbbbc84f5b9c9a7b467dc7b06e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:35.483352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcaa5b1-8039-477c-b71d-bcd25e0538ee None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.483880 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcaa5b1-8039-477c-b71d-bcd25e0538ee None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.484202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcaa5b1-8039-477c-b71d-bcd25e0538ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.503413 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-bbcaa5b1-8039-477c-b71d-bcd25e0538ee None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:35.504714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-bbcaa5b1-8039-477c-b71d-bcd25e0538ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:35.520686 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:35.522972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1231/2468] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:35.531936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ec19b-9383-480e-8cbd-30cfe2d9cb63 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.532204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ec19b-9383-480e-8cbd-30cfe2d9cb63 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.532490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ec19b-9383-480e-8cbd-30cfe2d9cb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.555380 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-954ec19b-9383-480e-8cbd-30cfe2d9cb63 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:35.556737 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-954ec19b-9383-480e-8cbd-30cfe2d9cb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:35.568257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1232/2469] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:35.572532 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ec19b-9383-480e-8cbd-30cfe2d9cb63 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:35.582606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1238/2470] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:35.589709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ec19b-9383-480e-8cbd-30cfe2d9cb63 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:35.589709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e741667-7bb7-4459-a7cc-f81ab010dc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:35.591233 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34800cfd-e2fb-4e3a-90f0-6870192ba349 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.591528 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34800cfd-e2fb-4e3a-90f0-6870192ba349 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.591884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34800cfd-e2fb-4e3a-90f0-6870192ba349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.602079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e741667-7bb7-4459-a7cc-f81ab010dc3e admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.602079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e741667-7bb7-4459-a7cc-f81ab010dc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.602079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e741667-7bb7-4459-a7cc-f81ab010dc3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.602079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e741667-7bb7-4459-a7cc-f81ab010dc3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:15:35.602079 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e741667-7bb7-4459-a7cc-f81ab010dc3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:35.603562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e741667-7bb7-4459-a7cc-f81ab010dc3e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:35.614541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-34800cfd-e2fb-4e3a-90f0-6870192ba349 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:35.616146 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-34800cfd-e2fb-4e3a-90f0-6870192ba349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:35.616269 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1233/2471] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:15:35.620346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e741667-7bb7-4459-a7cc-f81ab010dc3e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:35.628024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1239/2472] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:35.632058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34800cfd-e2fb-4e3a-90f0-6870192ba349 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:35.632895 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e741667-7bb7-4459-a7cc-f81ab010dc3e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:35.637682 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77a47a-fe1a-485b-81bb-0a37a0ebb8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:35.638401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77a47a-fe1a-485b-81bb-0a37a0ebb8ad admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.638692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77a47a-fe1a-485b-81bb-0a37a0ebb8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.638974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77a47a-fe1a-485b-81bb-0a37a0ebb8ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.640799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77a47a-fe1a-485b-81bb-0a37a0ebb8ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:35.642851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77a47a-fe1a-485b-81bb-0a37a0ebb8ad admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:35.643475 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34800cfd-e2fb-4e3a-90f0-6870192ba349 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:35.648217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12935f27-cfef-4732-8a69-e57bf5aa52b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:35.648882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12935f27-cfef-4732-8a69-e57bf5aa52b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.649168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12935f27-cfef-4732-8a69-e57bf5aa52b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.649449 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12935f27-cfef-4732-8a69-e57bf5aa52b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.650218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12935f27-cfef-4732-8a69-e57bf5aa52b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:15:35.650520 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12935f27-cfef-4732-8a69-e57bf5aa52b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:35.652142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12935f27-cfef-4732-8a69-e57bf5aa52b3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:35.665593 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1240/2473] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 10:15:35.669983 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12935f27-cfef-4732-8a69-e57bf5aa52b3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:35.680410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12935f27-cfef-4732-8a69-e57bf5aa52b3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:35.685847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1234/2474] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:35 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:35.687168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77a47a-fe1a-485b-81bb-0a37a0ebb8ad admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:35.740422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f7f37-8132-49fa-b546-a498ab5aabff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:35.741325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f7f37-8132-49fa-b546-a498ab5aabff admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.741610 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f7f37-8132-49fa-b546-a498ab5aabff admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.741995 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f7f37-8132-49fa-b546-a498ab5aabff admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.742678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f7f37-8132-49fa-b546-a498ab5aabff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:35.744613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f7f37-8132-49fa-b546-a498ab5aabff admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:35.765990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77a47a-fe1a-485b-81bb-0a37a0ebb8ad admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:35.771548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18067157-daff-402e-b32a-7336558923ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:35.772235 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18067157-daff-402e-b32a-7336558923ac admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.772534 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18067157-daff-402e-b32a-7336558923ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.772936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18067157-daff-402e-b32a-7336558923ac admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.773791 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18067157-daff-402e-b32a-7336558923ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:35.775767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18067157-daff-402e-b32a-7336558923ac admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:35.780731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1241/2475] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:35 2022] POST /identity/v3/projects => generated 361 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:35.782041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f7f37-8132-49fa-b546-a498ab5aabff admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:35.805188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1235/2476] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:35 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:35.806623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18067157-daff-402e-b32a-7336558923ac admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:35.820472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18067157-daff-402e-b32a-7336558923ac admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:35.856528 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f7f37-8132-49fa-b546-a498ab5aabff admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:35.865062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c0863-9f12-4890-99a5-b70b47b3f4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:35.865559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c0863-9f12-4890-99a5-b70b47b3f4d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.866071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c0863-9f12-4890-99a5-b70b47b3f4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.866210 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c0863-9f12-4890-99a5-b70b47b3f4d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.866841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c0863-9f12-4890-99a5-b70b47b3f4d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:35.869766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c0863-9f12-4890-99a5-b70b47b3f4d8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:35.896867 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5762914-5706-49c1-b4d2-bb0ffa47da6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:35.897651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5762914-5706-49c1-b4d2-bb0ffa47da6e admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.897965 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5762914-5706-49c1-b4d2-bb0ffa47da6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.898272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5762914-5706-49c1-b4d2-bb0ffa47da6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.899072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5762914-5706-49c1-b4d2-bb0ffa47da6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:35.900817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5762914-5706-49c1-b4d2-bb0ffa47da6e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:35.910315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:35.910809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1242/2477] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:35 2022] POST /identity/v3/users => generated 281 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:35.912658 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c0863-9f12-4890-99a5-b70b47b3f4d8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:35.918162 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1236/2478] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:35.923030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5762914-5706-49c1-b4d2-bb0ffa47da6e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:35.926600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c0863-9f12-4890-99a5-b70b47b3f4d8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:35.932127 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e9aa7-fe9f-4b04-8e2f-4603d6379c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:35.933117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e9aa7-fe9f-4b04-8e2f-4603d6379c92 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.933289 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e9aa7-fe9f-4b04-8e2f-4603d6379c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.933575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e9aa7-fe9f-4b04-8e2f-4603d6379c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.934519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e9aa7-fe9f-4b04-8e2f-4603d6379c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:35.935872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5762914-5706-49c1-b4d2-bb0ffa47da6e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:35.937229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e9aa7-fe9f-4b04-8e2f-4603d6379c92 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:35.948985 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1243/2479] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:35.949595 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd561c-1fcf-41bf-b259-0cf28684fde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:35.951723 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd561c-1fcf-41bf-b259-0cf28684fde2 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.952918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd561c-1fcf-41bf-b259-0cf28684fde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.953175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd561c-1fcf-41bf-b259-0cf28684fde2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.953722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e9aa7-fe9f-4b04-8e2f-4603d6379c92 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:35.954371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd561c-1fcf-41bf-b259-0cf28684fde2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:35.956451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd561c-1fcf-41bf-b259-0cf28684fde2 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:35.960737 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:35.966383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e9aa7-fe9f-4b04-8e2f-4603d6379c92 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:35.971514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1237/2480] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:35.976623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd561c-1fcf-41bf-b259-0cf28684fde2 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:35.977036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc75d3-d8aa-4b62-8854-054d0ff3e107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:35.977869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc75d3-d8aa-4b62-8854-054d0ff3e107 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.978272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc75d3-d8aa-4b62-8854-054d0ff3e107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.978648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc75d3-d8aa-4b62-8854-054d0ff3e107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:35.979480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc75d3-d8aa-4b62-8854-054d0ff3e107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:35.981197 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc75d3-d8aa-4b62-8854-054d0ff3e107 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:35.990770 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd561c-1fcf-41bf-b259-0cf28684fde2 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:35.998963 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1244/2481] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:15:35.999137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bd607-4df9-493b-8f07-c4f69eee0f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:35.999137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bd607-4df9-493b-8f07-c4f69eee0f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:35.999137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bd607-4df9-493b-8f07-c4f69eee0f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:35.999137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bd607-4df9-493b-8f07-c4f69eee0f48 admin admin] PATH_INFO: `/v3/projects/f25d559b4b6b41219ad36c5a3095ac28/users/1bada738d94b4424b76f8637a6aab66a/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:36.000702 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc75d3-d8aa-4b62-8854-054d0ff3e107 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:36.013730 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc75d3-d8aa-4b62-8854-054d0ff3e107 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:36.016003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bd607-4df9-493b-8f07-c4f69eee0f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f25d559b4b6b41219ad36c5a3095ac28, user_id=1bada738d94b4424b76f8637a6aab66a, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:36.019031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bd607-4df9-493b-8f07-c4f69eee0f48 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:36.020819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77567d61-b82c-438e-ab81-5928cd014a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:36.021756 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77567d61-b82c-438e-ab81-5928cd014a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:36.022089 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77567d61-b82c-438e-ab81-5928cd014a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:36.022314 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77567d61-b82c-438e-ab81-5928cd014a8e admin admin] PATH_INFO: `/v3/projects/d9200fc6e0d24b92a46d93617d6af083/users/900af484a5184af39423905e4fc5fe76/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:36.046822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1238/2482] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:35 2022] PUT /identity/v3/projects/f25d559b4b6b41219ad36c5a3095ac28/users/1bada738d94b4424b76f8637a6aab66a/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:36.055627 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77567d61-b82c-438e-ab81-5928cd014a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9200fc6e0d24b92a46d93617d6af083, user_id=900af484a5184af39423905e4fc5fe76, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:36.059966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefefcc-243e-4cb0-949d-760658c2f95b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:36.060458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefefcc-243e-4cb0-949d-760658c2f95b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:36.060898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefefcc-243e-4cb0-949d-760658c2f95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:36.066376 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77567d61-b82c-438e-ab81-5928cd014a8e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:36.091453 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1245/2483] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:35 2022] PUT /identity/v3/projects/d9200fc6e0d24b92a46d93617d6af083/users/900af484a5184af39423905e4fc5fe76/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:36.098044 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-6eefefcc-243e-4cb0-949d-760658c2f95b None None] MFA Rules not processed for user `1bada738d94b4424b76f8637a6aab66a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:36.099398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6eefefcc-243e-4cb0-949d-760658c2f95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:36.103022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cd43a-c4b7-4f06-93aa-4e1db9210cef None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:36.103316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cd43a-c4b7-4f06-93aa-4e1db9210cef None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:36.103597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cd43a-c4b7-4f06-93aa-4e1db9210cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:36.158819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ef7cd43a-c4b7-4f06-93aa-4e1db9210cef None None] MFA Rules not processed for user `900af484a5184af39423905e4fc5fe76`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:36.160607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ef7cd43a-c4b7-4f06-93aa-4e1db9210cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:36.225680 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1239/2484] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:36.237395 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefefcc-243e-4cb0-949d-760658c2f95b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:36.239136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6eefefcc-243e-4cb0-949d-760658c2f95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:36.241435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1246/2485] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:36.246768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cd43a-c4b7-4f06-93aa-4e1db9210cef None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:36.313560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cd43a-c4b7-4f06-93aa-4e1db9210cef None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:36.317040 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefefcc-243e-4cb0-949d-760658c2f95b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:36.318229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894c63f-8d6c-44f6-a545-46174e91184a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:36.318945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894c63f-8d6c-44f6-a545-46174e91184a admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:36.319878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894c63f-8d6c-44f6-a545-46174e91184a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:36.320160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894c63f-8d6c-44f6-a545-46174e91184a admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:36.320852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894c63f-8d6c-44f6-a545-46174e91184a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:36.322834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894c63f-8d6c-44f6-a545-46174e91184a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:36.324475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64bbfc-fb33-4b61-b1ee-4f49342c55e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:36.325156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64bbfc-fb33-4b61-b1ee-4f49342c55e1 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:36.325442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64bbfc-fb33-4b61-b1ee-4f49342c55e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:36.325749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64bbfc-fb33-4b61-b1ee-4f49342c55e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:36.334303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64bbfc-fb33-4b61-b1ee-4f49342c55e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:36.335898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64bbfc-fb33-4b61-b1ee-4f49342c55e1 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:36.345877 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1240/2486] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:36.355425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1247/2487] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:36 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:36.356451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64bbfc-fb33-4b61-b1ee-4f49342c55e1 service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:36.367355 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64bbfc-fb33-4b61-b1ee-4f49342c55e1 service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:36.405056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:36.426839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2cac5-a507-4a6c-aed2-b10e30b58f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:36.427523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2cac5-a507-4a6c-aed2-b10e30b58f9c admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:36.427811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2cac5-a507-4a6c-aed2-b10e30b58f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:36.428093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2cac5-a507-4a6c-aed2-b10e30b58f9c admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:36.428787 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2cac5-a507-4a6c-aed2-b10e30b58f9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:36.430799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2cac5-a507-4a6c-aed2-b10e30b58f9c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:36.452608 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1241/2488] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:36 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:36.456680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894c63f-8d6c-44f6-a545-46174e91184a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:36.468175 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894c63f-8d6c-44f6-a545-46174e91184a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:36.473834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d69d9-c8a2-480f-b715-318747a064fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:36.474648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d69d9-c8a2-480f-b715-318747a064fe admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:36.474946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d69d9-c8a2-480f-b715-318747a064fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:36.475445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d69d9-c8a2-480f-b715-318747a064fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:36.476218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d69d9-c8a2-480f-b715-318747a064fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:36.477907 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d69d9-c8a2-480f-b715-318747a064fe admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:36.491203 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1248/2489] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:36.495644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2cac5-a507-4a6c-aed2-b10e30b58f9c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:36.508331 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2cac5-a507-4a6c-aed2-b10e30b58f9c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:36.515296 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4904864-40d1-474d-b92a-e205674e38fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:36.516297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4904864-40d1-474d-b92a-e205674e38fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:36.516617 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4904864-40d1-474d-b92a-e205674e38fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:36.516945 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4904864-40d1-474d-b92a-e205674e38fd admin admin] PATH_INFO: `/v3/projects/854f419cc9b847069cd41e2b40b25cd8/users/6313638d18594d6f918c83429d5426db/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:36.536948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4904864-40d1-474d-b92a-e205674e38fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=854f419cc9b847069cd41e2b40b25cd8, user_id=6313638d18594d6f918c83429d5426db, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:36.539678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4904864-40d1-474d-b92a-e205674e38fd admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:36.559247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1242/2490] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:36 2022] PUT /identity/v3/projects/854f419cc9b847069cd41e2b40b25cd8/users/6313638d18594d6f918c83429d5426db/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:36.562935 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d69d9-c8a2-480f-b715-318747a064fe admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:36.579885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d69d9-c8a2-480f-b715-318747a064fe admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:36.632324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:36.690162 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6c88a-c48d-46cb-a011-82ee1ce944b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:36.690993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6c88a-c48d-46cb-a011-82ee1ce944b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:36.691341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6c88a-c48d-46cb-a011-82ee1ce944b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:36.691682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6c88a-c48d-46cb-a011-82ee1ce944b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:36.692535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6c88a-c48d-46cb-a011-82ee1ce944b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:36.694563 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6c88a-c48d-46cb-a011-82ee1ce944b4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:36.710104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1249/2491] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:36.711877 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4904864-40d1-474d-b92a-e205674e38fd admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:36.729881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4904864-40d1-474d-b92a-e205674e38fd admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:36.735659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac7af5-04af-44ac-bb2a-a229802cf979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:36.736823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac7af5-04af-44ac-bb2a-a229802cf979 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:36.737222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac7af5-04af-44ac-bb2a-a229802cf979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:36.737697 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac7af5-04af-44ac-bb2a-a229802cf979 admin admin] PATH_INFO: `/v3/projects/854f419cc9b847069cd41e2b40b25cd8/users/6313638d18594d6f918c83429d5426db/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:36.740055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac7af5-04af-44ac-bb2a-a229802cf979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=854f419cc9b847069cd41e2b40b25cd8, user_id=6313638d18594d6f918c83429d5426db, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:36.742995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac7af5-04af-44ac-bb2a-a229802cf979 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:36.765822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1243/2492] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:36 2022] PUT /identity/v3/projects/854f419cc9b847069cd41e2b40b25cd8/users/6313638d18594d6f918c83429d5426db/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:36.767889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e90462-4227-4f40-b70c-6925c24f455c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:36.768387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e90462-4227-4f40-b70c-6925c24f455c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:36.768897 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e90462-4227-4f40-b70c-6925c24f455c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:36.808325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b7e90462-4227-4f40-b70c-6925c24f455c None None] MFA Rules not processed for user `6313638d18594d6f918c83429d5426db`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:36.809761 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b7e90462-4227-4f40-b70c-6925c24f455c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:36.884604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1250/2493] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:36 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:36.896265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac7af5-04af-44ac-bb2a-a229802cf979 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:36.898482 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-dbac7af5-04af-44ac-bb2a-a229802cf979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:36.960250 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac7af5-04af-44ac-bb2a-a229802cf979 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:36.964367 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1fb02-b820-4e63-a3d2-f05e4b42f589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:36.965019 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1fb02-b820-4e63-a3d2-f05e4b42f589 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:36.965306 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1fb02-b820-4e63-a3d2-f05e4b42f589 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:36.965584 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1fb02-b820-4e63-a3d2-f05e4b42f589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:36.975314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1fb02-b820-4e63-a3d2-f05e4b42f589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:36.976758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1fb02-b820-4e63-a3d2-f05e4b42f589 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:36.987317 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1244/2494] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:36 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:15:37.081665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e90462-4227-4f40-b70c-6925c24f455c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:37.092420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e90462-4227-4f40-b70c-6925c24f455c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:37.096922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fb8fc-f6f9-4dd0-9b56-2237f80046ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:37.097579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fb8fc-f6f9-4dd0-9b56-2237f80046ed service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:37.097907 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fb8fc-f6f9-4dd0-9b56-2237f80046ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:37.098188 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fb8fc-f6f9-4dd0-9b56-2237f80046ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:37.157938 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fb8fc-f6f9-4dd0-9b56-2237f80046ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:37.159398 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fb8fc-f6f9-4dd0-9b56-2237f80046ed service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:37.168164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1251/2495] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:40.970333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1fb02-b820-4e63-a3d2-f05e4b42f589 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:41.035972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1fb02-b820-4e63-a3d2-f05e4b42f589 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:41.040800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:41.041849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:41.042183 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:41.042418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] PATH_INFO: `/v3/users/900af484a5184af39423905e4fc5fe76` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:41.043614 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=900af484a5184af39423905e4fc5fe76)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:41.045549 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:41.087523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:41.098017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '900af484a5184af39423905e4fc5fe76'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.098723 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '900af484a5184af39423905e4fc5fe76'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.107184 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '900af484a5184af39423905e4fc5fe76'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.119679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '900af484a5184af39423905e4fc5fe76'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.120415 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '900af484a5184af39423905e4fc5fe76'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.134516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1245/2496] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:40 2022] DELETE /identity/v3/users/900af484a5184af39423905e4fc5fe76 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:41.139476 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0b742-b845-47a0-9d30-0393d38d62f3 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:41.140005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0b742-b845-47a0-9d30-0393d38d62f3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:41.140005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0b742-b845-47a0-9d30-0393d38d62f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:41.158541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f6b0b742-b845-47a0-9d30-0393d38d62f3 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:41.160051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f6b0b742-b845-47a0-9d30-0393d38d62f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:41.211296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:41.225590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1252/2497] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:41.234796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:41.236339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:41.261410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0b742-b845-47a0-9d30-0393d38d62f3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:41.262777 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f6b0b742-b845-47a0-9d30-0393d38d62f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:41.275130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0b742-b845-47a0-9d30-0393d38d62f3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:41.279917 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:41.280821 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:41.281106 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:41.281385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] PATH_INFO: `/v3/users/1bada738d94b4424b76f8637a6aab66a` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:41.282636 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bada738d94b4424b76f8637a6aab66a)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:41.284937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:41.296232 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4b42a-22cc-4a7d-a0d7-a03b0da38a06 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:41.300204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf33e82-5776-43d7-a119-36805ae956a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:41.300855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf33e82-5776-43d7-a119-36805ae956a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:41.301210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf33e82-5776-43d7-a119-36805ae956a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:41.301493 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf33e82-5776-43d7-a119-36805ae956a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:41.310659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf33e82-5776-43d7-a119-36805ae956a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:41.312134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf33e82-5776-43d7-a119-36805ae956a9 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:41.323369 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1246/2498] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:41.334126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bada738d94b4424b76f8637a6aab66a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.334832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bada738d94b4424b76f8637a6aab66a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.343174 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bada738d94b4424b76f8637a6aab66a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.356693 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bada738d94b4424b76f8637a6aab66a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.357465 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bada738d94b4424b76f8637a6aab66a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.372530 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1253/2499] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:41 2022] DELETE /identity/v3/users/1bada738d94b4424b76f8637a6aab66a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:41.377723 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d5d48-787f-4b3c-9afe-5af02ae9030c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:41.378046 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d5d48-787f-4b3c-9afe-5af02ae9030c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:41.378311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d5d48-787f-4b3c-9afe-5af02ae9030c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:41.398019 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c61d5d48-787f-4b3c-9afe-5af02ae9030c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:41.399353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c61d5d48-787f-4b3c-9afe-5af02ae9030c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:41.465158 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1247/2500] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:41.475614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:41.477422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:41.539505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f9773-be33-41fb-8dc5-6288a7ebddc0 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:41.544312 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120366e-f293-44e6-a399-d4cac62cba20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:41.545036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120366e-f293-44e6-a399-d4cac62cba20 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:41.545388 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120366e-f293-44e6-a399-d4cac62cba20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:41.545763 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120366e-f293-44e6-a399-d4cac62cba20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:41.556500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120366e-f293-44e6-a399-d4cac62cba20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:41.558112 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120366e-f293-44e6-a399-d4cac62cba20 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:41.577770 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1254/2501] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:41.619002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d5d48-787f-4b3c-9afe-5af02ae9030c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:41.620560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c61d5d48-787f-4b3c-9afe-5af02ae9030c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:41.633300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d5d48-787f-4b3c-9afe-5af02ae9030c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:41.639518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:41.640580 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:41.640943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:41.641191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] PATH_INFO: `/v3/projects/d9200fc6e0d24b92a46d93617d6af083` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:41.642436 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9200fc6e0d24b92a46d93617d6af083)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:41.644506 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:41.708246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9200fc6e0d24b92a46d93617d6af083', 'request_id': 'req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.711069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:41.718015 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9200fc6e0d24b92a46d93617d6af083', 'request_id': 'req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.732654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9200fc6e0d24b92a46d93617d6af083', 'request_id': 'req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.759873 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] The token cache is being invalidate because project d9200fc6e0d24b92a46d93617d6af083 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:41.760171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.766095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.767791 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1248/2502] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:41 2022] DELETE /identity/v3/projects/d9200fc6e0d24b92a46d93617d6af083 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:41.772572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120366e-f293-44e6-a399-d4cac62cba20 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:41.787653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120366e-f293-44e6-a399-d4cac62cba20 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:41.865109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:41.866441 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:41.867377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:41.868015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] PATH_INFO: `/v3/users/6313638d18594d6f918c83429d5426db` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:41.869802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6313638d18594d6f918c83429d5426db)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:41.872698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:41.896124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:41.913533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:41.915181 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:41.934112 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b63c2-dd05-4392-a12d-7c3c01d7fcea admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:41.940423 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621dfa8-3d08-4da1-b3a4-e43b76b9d374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:41.942340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621dfa8-3d08-4da1-b3a4-e43b76b9d374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:41.942627 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621dfa8-3d08-4da1-b3a4-e43b76b9d374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:41.942948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621dfa8-3d08-4da1-b3a4-e43b76b9d374 admin admin] PATH_INFO: `/v3/users/5f136228a90f4527b6c8de2bafe57a70` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:41.944879 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621dfa8-3d08-4da1-b3a4-e43b76b9d374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f136228a90f4527b6c8de2bafe57a70)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:41.947069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621dfa8-3d08-4da1-b3a4-e43b76b9d374 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:41.971929 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6313638d18594d6f918c83429d5426db'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.972859 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6313638d18594d6f918c83429d5426db'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:41.988648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6313638d18594d6f918c83429d5426db'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.003438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6313638d18594d6f918c83429d5426db'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.004243 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6313638d18594d6f918c83429d5426db'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.018943 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1255/2503] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:41 2022] DELETE /identity/v3/users/6313638d18594d6f918c83429d5426db => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:42.022774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:42.024125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.025488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d621dfa8-3d08-4da1-b3a4-e43b76b9d374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f136228a90f4527b6c8de2bafe57a70'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.026775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d621dfa8-3d08-4da1-b3a4-e43b76b9d374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f136228a90f4527b6c8de2bafe57a70'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.036882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d621dfa8-3d08-4da1-b3a4-e43b76b9d374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f136228a90f4527b6c8de2bafe57a70'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.050840 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d621dfa8-3d08-4da1-b3a4-e43b76b9d374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f136228a90f4527b6c8de2bafe57a70'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.052042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d621dfa8-3d08-4da1-b3a4-e43b76b9d374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f136228a90f4527b6c8de2bafe57a70'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.066379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1249/2504] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:41 2022] DELETE /identity/v3/users/5f136228a90f4527b6c8de2bafe57a70 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:42.070884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1278c-2f31-47e7-886e-165fd88b81d6 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.071184 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1278c-2f31-47e7-886e-165fd88b81d6 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.071459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1278c-2f31-47e7-886e-165fd88b81d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.090465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d9f1278c-2f31-47e7-886e-165fd88b81d6 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:42.091814 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d9f1278c-2f31-47e7-886e-165fd88b81d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.098355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b2739-2742-4570-944a-c60c3b6d1d65 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:42.167400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:42.168272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.168591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.168871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d admin admin] PATH_INFO: `/v3/projects/f25d559b4b6b41219ad36c5a3095ac28` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.170104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f25d559b4b6b41219ad36c5a3095ac28)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:42.172038 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:42.174031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1250/2505] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:42.188987 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1278c-2f31-47e7-886e-165fd88b81d6 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:42.193379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d9f1278c-2f31-47e7-886e-165fd88b81d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.225095 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:42.242048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f25d559b4b6b41219ad36c5a3095ac28', 'request_id': 'req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.249243 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f25d559b4b6b41219ad36c5a3095ac28', 'request_id': 'req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.256225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f25d559b4b6b41219ad36c5a3095ac28', 'request_id': 'req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.260842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1278c-2f31-47e7-886e-165fd88b81d6 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:42.266731 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:42.275551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d admin admin] The token cache is being invalidate because project f25d559b4b6b41219ad36c5a3095ac28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:42.275962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.276769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8df901c6-33c3-4e10-b65b-cf277dbe7e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.282230 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1256/2506] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:42 2022] DELETE /identity/v3/projects/f25d559b4b6b41219ad36c5a3095ac28 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:42.284118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13460456-caac-4b89-bb9d-bc052b967fea None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.284409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13460456-caac-4b89-bb9d-bc052b967fea None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.284683 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13460456-caac-4b89-bb9d-bc052b967fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.305053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-13460456-caac-4b89-bb9d-bc052b967fea None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:42.305053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-13460456-caac-4b89-bb9d-bc052b967fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.326102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c667f2e-fd7e-4d15-ab0d-734afa321747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:42.326780 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c667f2e-fd7e-4d15-ab0d-734afa321747 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.327256 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c667f2e-fd7e-4d15-ab0d-734afa321747 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.327578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c667f2e-fd7e-4d15-ab0d-734afa321747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.329713 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9c667f2e-fd7e-4d15-ab0d-734afa321747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.383949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1257/2507] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:42.390631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13460456-caac-4b89-bb9d-bc052b967fea None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:42.392044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-13460456-caac-4b89-bb9d-bc052b967fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.395218 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c667f2e-fd7e-4d15-ab0d-734afa321747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:42.396694 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c667f2e-fd7e-4d15-ab0d-734afa321747 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:42.404054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13460456-caac-4b89-bb9d-bc052b967fea None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:42.406825 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1251/2508] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:42.408150 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324f26e-365b-4ba8-9c16-dc3e12f702a5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.408464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324f26e-365b-4ba8-9c16-dc3e12f702a5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.408742 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:42.408871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324f26e-365b-4ba8-9c16-dc3e12f702a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.409769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.410054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.410323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc admin admin] PATH_INFO: `/v3/projects/854f419cc9b847069cd41e2b40b25cd8` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.411497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=854f419cc9b847069cd41e2b40b25cd8)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:42.413405 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:42.426630 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c324f26e-365b-4ba8-9c16-dc3e12f702a5 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:42.428026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c324f26e-365b-4ba8-9c16-dc3e12f702a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.436427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1252/2509] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:42.440511 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324f26e-365b-4ba8-9c16-dc3e12f702a5 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:42.450807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324f26e-365b-4ba8-9c16-dc3e12f702a5 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:42.456966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51614b0c-edb2-45a4-8b2c-e126156814c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:42.457596 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51614b0c-edb2-45a4-8b2c-e126156814c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.457913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51614b0c-edb2-45a4-8b2c-e126156814c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.458190 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51614b0c-edb2-45a4-8b2c-e126156814c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.458961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51614b0c-edb2-45a4-8b2c-e126156814c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:15:42.459306 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51614b0c-edb2-45a4-8b2c-e126156814c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:42.461208 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51614b0c-edb2-45a4-8b2c-e126156814c6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:42.476220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1253/2510] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 10:15:42.480996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51614b0c-edb2-45a4-8b2c-e126156814c6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:42.484465 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854f419cc9b847069cd41e2b40b25cd8', 'request_id': 'req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.492039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51614b0c-edb2-45a4-8b2c-e126156814c6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:42.492990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854f419cc9b847069cd41e2b40b25cd8', 'request_id': 'req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.500643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854f419cc9b847069cd41e2b40b25cd8', 'request_id': 'req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.517594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc admin admin] The token cache is being invalidate because project 854f419cc9b847069cd41e2b40b25cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:42.518164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.518957 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fbd04e36-3089-4ec7-9ccd-ca534fff52cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.520567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1258/2511] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:42 2022] DELETE /identity/v3/projects/854f419cc9b847069cd41e2b40b25cd8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:42.526150 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7e09a-122f-4752-9070-6c5d227114e2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.526403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7e09a-122f-4752-9070-6c5d227114e2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.526840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7e09a-122f-4752-9070-6c5d227114e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.545655 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-39d7e09a-122f-4752-9070-6c5d227114e2 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:42.546861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-39d7e09a-122f-4752-9070-6c5d227114e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.561491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb6b40-6bf9-4fc0-8d7a-c805544eafb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:42.562299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb6b40-6bf9-4fc0-8d7a-c805544eafb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.562587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb6b40-6bf9-4fc0-8d7a-c805544eafb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.562864 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb6b40-6bf9-4fc0-8d7a-c805544eafb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.566520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb6b40-6bf9-4fc0-8d7a-c805544eafb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:42.566520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb6b40-6bf9-4fc0-8d7a-c805544eafb6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:42.598284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:42.605660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1254/2512] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:42 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:42.607948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb6b40-6bf9-4fc0-8d7a-c805544eafb6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:42.610001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6feb6b40-6bf9-4fc0-8d7a-c805544eafb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.619495 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1259/2513] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:42.626713 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc0021-3d2d-482e-97bc-8fa3e2b7c1e8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.627003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc0021-3d2d-482e-97bc-8fa3e2b7c1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.627485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc0021-3d2d-482e-97bc-8fa3e2b7c1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.644845 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:42.651079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-31cc0021-3d2d-482e-97bc-8fa3e2b7c1e8 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:42.652836 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-31cc0021-3d2d-482e-97bc-8fa3e2b7c1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.698675 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb6b40-6bf9-4fc0-8d7a-c805544eafb6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:42.705169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c69107-c412-4494-a346-7c56101a4652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:42.706194 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c69107-c412-4494-a346-7c56101a4652 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.706504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c69107-c412-4494-a346-7c56101a4652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.706839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c69107-c412-4494-a346-7c56101a4652 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.707667 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c69107-c412-4494-a346-7c56101a4652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:42.709717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c69107-c412-4494-a346-7c56101a4652 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:42.726631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1260/2514] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:42.727716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc0021-3d2d-482e-97bc-8fa3e2b7c1e8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:42.729156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-31cc0021-3d2d-482e-97bc-8fa3e2b7c1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.733990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1255/2515] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:42 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:42.734794 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c69107-c412-4494-a346-7c56101a4652 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:42.742592 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc0021-3d2d-482e-97bc-8fa3e2b7c1e8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:42.746137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c69107-c412-4494-a346-7c56101a4652 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:42.747507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:42.748395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.748670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.748972 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] PATH_INFO: `/v3/projects/d9e22331c94e45a7aab639eef31482c2` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.750184 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e22331c94e45a7aab639eef31482c2)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:42.752139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:42.752312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d3931-32fa-4eee-b642-73fdd2754391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:42.753088 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d3931-32fa-4eee-b642-73fdd2754391 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.753427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d3931-32fa-4eee-b642-73fdd2754391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.753647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d3931-32fa-4eee-b642-73fdd2754391 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.754512 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d3931-32fa-4eee-b642-73fdd2754391 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:15:42.754812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d3931-32fa-4eee-b642-73fdd2754391 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:42.756410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d3931-32fa-4eee-b642-73fdd2754391 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:42.769687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1256/2516] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:15:42.770485 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d3931-32fa-4eee-b642-73fdd2754391 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:42.780542 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d3931-32fa-4eee-b642-73fdd2754391 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:42.785842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22bca2-5b2a-4e2b-839e-f6950e90530c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:42.786705 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22bca2-5b2a-4e2b-839e-f6950e90530c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.787058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22bca2-5b2a-4e2b-839e-f6950e90530c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.787781 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22bca2-5b2a-4e2b-839e-f6950e90530c admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.788517 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22bca2-5b2a-4e2b-839e-f6950e90530c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:42.790180 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22bca2-5b2a-4e2b-839e-f6950e90530c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:42.800381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1257/2517] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:42.801151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22bca2-5b2a-4e2b-839e-f6950e90530c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:42.802904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e22331c94e45a7aab639eef31482c2', 'request_id': 'req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.808505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e22331c94e45a7aab639eef31482c2', 'request_id': 'req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.812463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22bca2-5b2a-4e2b-839e-f6950e90530c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:42.814717 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e22331c94e45a7aab639eef31482c2', 'request_id': 'req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.829741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] The token cache is being invalidate because project d9e22331c94e45a7aab639eef31482c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:42.830094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.830705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:42.832349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1261/2518] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:42 2022] DELETE /identity/v3/projects/d9e22331c94e45a7aab639eef31482c2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:42.833259 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:42.834630 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.872330 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5ae59-0020-4c92-9e96-1fd06ee8a021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:42.873017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5ae59-0020-4c92-9e96-1fd06ee8a021 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.873315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5ae59-0020-4c92-9e96-1fd06ee8a021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.873595 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5ae59-0020-4c92-9e96-1fd06ee8a021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.874315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5ae59-0020-4c92-9e96-1fd06ee8a021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:42.876262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5ae59-0020-4c92-9e96-1fd06ee8a021 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:42.905074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e082e-6bc0-4f4d-9564-e13ee6d1ccd1 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:42.907868 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1258/2519] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:42 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:42.908713 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5ae59-0020-4c92-9e96-1fd06ee8a021 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:42.910189 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106bf36-dc19-4cb0-8c44-2784048b45b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:42.910589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-caf5ae59-0020-4c92-9e96-1fd06ee8a021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.911236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106bf36-dc19-4cb0-8c44-2784048b45b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.911544 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106bf36-dc19-4cb0-8c44-2784048b45b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.911827 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106bf36-dc19-4cb0-8c44-2784048b45b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.912568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106bf36-dc19-4cb0-8c44-2784048b45b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:42.914243 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106bf36-dc19-4cb0-8c44-2784048b45b5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:42.925225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1262/2520] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:42.926225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106bf36-dc19-4cb0-8c44-2784048b45b5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:42.928091 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2106bf36-dc19-4cb0-8c44-2784048b45b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:42.941674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106bf36-dc19-4cb0-8c44-2784048b45b5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:42.947553 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:42.979840 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5ae59-0020-4c92-9e96-1fd06ee8a021 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:42.985320 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:42.986275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:42.986661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:42.986911 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] PATH_INFO: `/v3/users/ae6f36ef60de4ff7b964e464dc675f3f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:42.988557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae6f36ef60de4ff7b964e464dc675f3f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:42.990509 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.008576 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:43.011256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a98e0-0bf0-4604-af4b-f3c153a99464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.011612 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a98e0-0bf0-4604-af4b-f3c153a99464 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.011989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a98e0-0bf0-4604-af4b-f3c153a99464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.012349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a98e0-0bf0-4604-af4b-f3c153a99464 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.013028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a98e0-0bf0-4604-af4b-f3c153a99464 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.015144 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a98e0-0bf0-4604-af4b-f3c153a99464 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.043515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1263/2521] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:42 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:43.043515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a98e0-0bf0-4604-af4b-f3c153a99464 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.049892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae6f36ef60de4ff7b964e464dc675f3f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:43.050555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae6f36ef60de4ff7b964e464dc675f3f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:43.050757 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a98e0-0bf0-4604-af4b-f3c153a99464 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.055448 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23b327-a22f-49ed-9724-94deb47b19eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.056447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23b327-a22f-49ed-9724-94deb47b19eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.056728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23b327-a22f-49ed-9724-94deb47b19eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.057027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23b327-a22f-49ed-9724-94deb47b19eb admin admin] PATH_INFO: `/v3/projects/17ccee846512409fb020914db8d8a13b/users/baf5475348da4210a8fef2e11407b25e/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.059469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae6f36ef60de4ff7b964e464dc675f3f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:43.071281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae6f36ef60de4ff7b964e464dc675f3f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:43.071935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae6f36ef60de4ff7b964e464dc675f3f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:43.072053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23b327-a22f-49ed-9724-94deb47b19eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ccee846512409fb020914db8d8a13b, user_id=baf5475348da4210a8fef2e11407b25e, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.074726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23b327-a22f-49ed-9724-94deb47b19eb admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.084553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1259/2522] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:42 2022] DELETE /identity/v3/users/ae6f36ef60de4ff7b964e464dc675f3f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:43.085588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.087220 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:43.089208 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1264/2523] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:43 2022] PUT /identity/v3/projects/17ccee846512409fb020914db8d8a13b/users/baf5475348da4210a8fef2e11407b25e/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:43.101531 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7f9a8-b0bf-42f7-8d69-587b8e60ee6e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.101891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7f9a8-b0bf-42f7-8d69-587b8e60ee6e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.102265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7f9a8-b0bf-42f7-8d69-587b8e60ee6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.140445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-91a7f9a8-b0bf-42f7-8d69-587b8e60ee6e None None] MFA Rules not processed for user `baf5475348da4210a8fef2e11407b25e`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:43.141874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-91a7f9a8-b0bf-42f7-8d69-587b8e60ee6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:43.168888 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c58642-0b39-4b20-a9c7-48c1817a1dd7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.174477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729a849-f528-4e66-b80e-f9f0e3f3d115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.175180 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729a849-f528-4e66-b80e-f9f0e3f3d115 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.175454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729a849-f528-4e66-b80e-f9f0e3f3d115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.175738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729a849-f528-4e66-b80e-f9f0e3f3d115 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.176478 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729a849-f528-4e66-b80e-f9f0e3f3d115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.178153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729a849-f528-4e66-b80e-f9f0e3f3d115 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.189103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1260/2524] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:43.193771 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729a849-f528-4e66-b80e-f9f0e3f3d115 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.204334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729a849-f528-4e66-b80e-f9f0e3f3d115 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.210464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca378696-9c42-4b48-a787-6a00d6fff5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.211580 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca378696-9c42-4b48-a787-6a00d6fff5da admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.212013 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca378696-9c42-4b48-a787-6a00d6fff5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.213290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca378696-9c42-4b48-a787-6a00d6fff5da admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.214148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca378696-9c42-4b48-a787-6a00d6fff5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.215797 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca378696-9c42-4b48-a787-6a00d6fff5da admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.222321 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1265/2525] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:43.226830 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7f9a8-b0bf-42f7-8d69-587b8e60ee6e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.228890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-91a7f9a8-b0bf-42f7-8d69-587b8e60ee6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:43.230632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1261/2526] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:15:43.234648 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca378696-9c42-4b48-a787-6a00d6fff5da admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.247082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7f9a8-b0bf-42f7-8d69-587b8e60ee6e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.248150 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca378696-9c42-4b48-a787-6a00d6fff5da admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.251976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2da1d3-f394-49d3-b7f0-145eb6dec5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.253265 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2da1d3-f394-49d3-b7f0-145eb6dec5a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.253681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2da1d3-f394-49d3-b7f0-145eb6dec5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.254039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2da1d3-f394-49d3-b7f0-145eb6dec5a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.255008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2da1d3-f394-49d3-b7f0-145eb6dec5a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.256248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac251e3-6b50-47a3-8cc7-464ca9546d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.257277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2da1d3-f394-49d3-b7f0-145eb6dec5a6 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.257988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac251e3-6b50-47a3-8cc7-464ca9546d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.258411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac251e3-6b50-47a3-8cc7-464ca9546d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.259154 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac251e3-6b50-47a3-8cc7-464ca9546d18 admin admin] PATH_INFO: `/v3/projects/48666f7f79d249df9c54060e1bdf3839/users/9817b0c1cce549518d11fc09b57de891/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.277890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac251e3-6b50-47a3-8cc7-464ca9546d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48666f7f79d249df9c54060e1bdf3839, user_id=9817b0c1cce549518d11fc09b57de891, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.280953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac251e3-6b50-47a3-8cc7-464ca9546d18 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.296567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1266/2527] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:43 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:43.297492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2da1d3-f394-49d3-b7f0-145eb6dec5a6 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.299383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1262/2528] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:43 2022] PUT /identity/v3/projects/48666f7f79d249df9c54060e1bdf3839/users/9817b0c1cce549518d11fc09b57de891/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:43.311005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1651be-1400-45ca-911c-cc2695a188a1 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.311293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1651be-1400-45ca-911c-cc2695a188a1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.311560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1651be-1400-45ca-911c-cc2695a188a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.343238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5b1651be-1400-45ca-911c-cc2695a188a1 None None] MFA Rules not processed for user `9817b0c1cce549518d11fc09b57de891`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:43.344573 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5b1651be-1400-45ca-911c-cc2695a188a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:43.355409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:43.381846 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2da1d3-f394-49d3-b7f0-145eb6dec5a6 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.387779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8b2d5-301f-41f2-991f-e1f95a6ad332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.388123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8b2d5-301f-41f2-991f-e1f95a6ad332 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.388450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8b2d5-301f-41f2-991f-e1f95a6ad332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.388740 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8b2d5-301f-41f2-991f-e1f95a6ad332 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.389609 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8b2d5-301f-41f2-991f-e1f95a6ad332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.391911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8b2d5-301f-41f2-991f-e1f95a6ad332 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.395091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:43.420339 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1267/2529] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:43 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:43.421186 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8b2d5-301f-41f2-991f-e1f95a6ad332 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.424704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-beb8b2d5-301f-41f2-991f-e1f95a6ad332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:43.437237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8b2d5-301f-41f2-991f-e1f95a6ad332 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.442158 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.443029 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.443316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.443591 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] PATH_INFO: `/v3/projects/bd9f6859854d42a8987c2c203ecfe2f4` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.444908 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd9f6859854d42a8987c2c203ecfe2f4)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.445262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1263/2530] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:43.446223 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1651be-1400-45ca-911c-cc2695a188a1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.447011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.457366 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1651be-1400-45ca-911c-cc2695a188a1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.462032 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ceaf5-449e-4f32-8583-53b19769cb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.463035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ceaf5-449e-4f32-8583-53b19769cb26 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.463457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ceaf5-449e-4f32-8583-53b19769cb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.463734 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ceaf5-449e-4f32-8583-53b19769cb26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.464463 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ceaf5-449e-4f32-8583-53b19769cb26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.466096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ceaf5-449e-4f32-8583-53b19769cb26 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.476669 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1264/2531] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:43.477479 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ceaf5-449e-4f32-8583-53b19769cb26 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.487646 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ceaf5-449e-4f32-8583-53b19769cb26 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.495896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd9f6859854d42a8987c2c203ecfe2f4', 'request_id': 'req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:43.502329 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd9f6859854d42a8987c2c203ecfe2f4', 'request_id': 'req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:43.509178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd9f6859854d42a8987c2c203ecfe2f4', 'request_id': 'req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:43.523507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] The token cache is being invalidate because project bd9f6859854d42a8987c2c203ecfe2f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:43.523823 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:43.524478 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:43.526080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1268/2532] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:43 2022] DELETE /identity/v3/projects/bd9f6859854d42a8987c2c203ecfe2f4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:43.526944 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.528384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:43.548342 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28281b2f-0694-408a-8a4c-d71ccd25bcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.549033 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28281b2f-0694-408a-8a4c-d71ccd25bcb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.549343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28281b2f-0694-408a-8a4c-d71ccd25bcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.549678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28281b2f-0694-408a-8a4c-d71ccd25bcb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.550502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28281b2f-0694-408a-8a4c-d71ccd25bcb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.552433 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28281b2f-0694-408a-8a4c-d71ccd25bcb5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.582821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1265/2533] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:43 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:43.582821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad1bd3-7236-49fa-9242-b281ec006d52 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.582821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad1bd3-7236-49fa-9242-b281ec006d52 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.582821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad1bd3-7236-49fa-9242-b281ec006d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.593613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc935982-f932-4c36-8cb9-4b7fc9fe64f7 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.599836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4bad1bd3-7236-49fa-9242-b281ec006d52 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:43.601073 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4bad1bd3-7236-49fa-9242-b281ec006d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:43.650418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a156e8f-1edb-400a-8699-1612d23119ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.651320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a156e8f-1edb-400a-8699-1612d23119ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.651620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a156e8f-1edb-400a-8699-1612d23119ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.651877 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a156e8f-1edb-400a-8699-1612d23119ca admin admin] PATH_INFO: `/v3/projects/444c9611ae9f4260b580c5814d9d8d3e/users/9a38442b69bf433fa8d3d3a7b14d49e6/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.655419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:43.660726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1266/2534] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:43.667432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad1bd3-7236-49fa-9242-b281ec006d52 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.667432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4bad1bd3-7236-49fa-9242-b281ec006d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:43.673762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a156e8f-1edb-400a-8699-1612d23119ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=444c9611ae9f4260b580c5814d9d8d3e, user_id=9a38442b69bf433fa8d3d3a7b14d49e6, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.675919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad1bd3-7236-49fa-9242-b281ec006d52 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.676571 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a156e8f-1edb-400a-8699-1612d23119ca admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.680650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd568e-1090-4659-83af-061c26c6da08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.681778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd568e-1090-4659-83af-061c26c6da08 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.682148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd568e-1090-4659-83af-061c26c6da08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.682489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd568e-1090-4659-83af-061c26c6da08 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.683227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd568e-1090-4659-83af-061c26c6da08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.685461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd568e-1090-4659-83af-061c26c6da08 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.694403 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1269/2535] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:43 2022] PUT /identity/v3/projects/444c9611ae9f4260b580c5814d9d8d3e/users/9a38442b69bf433fa8d3d3a7b14d49e6/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:43.695685 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd8eb6-5cb8-4a03-8636-522e01b5bb8e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.695974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd8eb6-5cb8-4a03-8636-522e01b5bb8e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.696274 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd8eb6-5cb8-4a03-8636-522e01b5bb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.710045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1267/2536] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:43 2022] POST /identity/v3/users => generated 291 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:43.710708 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd568e-1090-4659-83af-061c26c6da08 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.715193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-06fd8eb6-5cb8-4a03-8636-522e01b5bb8e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:43.716391 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-06fd8eb6-5cb8-4a03-8636-522e01b5bb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:43.720657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd568e-1090-4659-83af-061c26c6da08 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.761723 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:43.777731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1270/2537] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:43.778649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd8eb6-5cb8-4a03-8636-522e01b5bb8e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.778772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e375d24-adcb-4f70-a5a7-d5c4ccd6a50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.779400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e375d24-adcb-4f70-a5a7-d5c4ccd6a50f admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.779715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e375d24-adcb-4f70-a5a7-d5c4ccd6a50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.779994 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e375d24-adcb-4f70-a5a7-d5c4ccd6a50f admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.780725 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e375d24-adcb-4f70-a5a7-d5c4ccd6a50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.782384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e375d24-adcb-4f70-a5a7-d5c4ccd6a50f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.788857 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd8eb6-5cb8-4a03-8636-522e01b5bb8e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.792802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1268/2538] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:43.793488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e375d24-adcb-4f70-a5a7-d5c4ccd6a50f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.797161 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015be785-bc63-4987-a1c0-55a6332aa1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.797860 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015be785-bc63-4987-a1c0-55a6332aa1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.798149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015be785-bc63-4987-a1c0-55a6332aa1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.798422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015be785-bc63-4987-a1c0-55a6332aa1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.799189 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015be785-bc63-4987-a1c0-55a6332aa1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.800927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015be785-bc63-4987-a1c0-55a6332aa1b3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.803293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e375d24-adcb-4f70-a5a7-d5c4ccd6a50f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.808127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3b349-15a1-4d6a-91a9-4c32a461f37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.808767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3b349-15a1-4d6a-91a9-4c32a461f37d admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.809050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3b349-15a1-4d6a-91a9-4c32a461f37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.809326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3b349-15a1-4d6a-91a9-4c32a461f37d admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.810080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3b349-15a1-4d6a-91a9-4c32a461f37d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:15:43.810385 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3b349-15a1-4d6a-91a9-4c32a461f37d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.814804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1271/2539] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:43.815006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3b349-15a1-4d6a-91a9-4c32a461f37d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.815140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015be785-bc63-4987-a1c0-55a6332aa1b3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.822280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015be785-bc63-4987-a1c0-55a6332aa1b3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.824452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1269/2540] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:15:43.825239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3b349-15a1-4d6a-91a9-4c32a461f37d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.827045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4b8cc-8eda-4c87-8ee7-78cb537909ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.827914 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4b8cc-8eda-4c87-8ee7-78cb537909ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.828200 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4b8cc-8eda-4c87-8ee7-78cb537909ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.828472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4b8cc-8eda-4c87-8ee7-78cb537909ed admin admin] PATH_INFO: `/v3/projects/444c9611ae9f4260b580c5814d9d8d3e/users/9a38442b69bf433fa8d3d3a7b14d49e6/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.830481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4b8cc-8eda-4c87-8ee7-78cb537909ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=444c9611ae9f4260b580c5814d9d8d3e, user_id=9a38442b69bf433fa8d3d3a7b14d49e6, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.833108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4b8cc-8eda-4c87-8ee7-78cb537909ed admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.837152 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3b349-15a1-4d6a-91a9-4c32a461f37d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.843751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1272/2541] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:43 2022] PUT /identity/v3/projects/444c9611ae9f4260b580c5814d9d8d3e/users/9a38442b69bf433fa8d3d3a7b14d49e6/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:43.844497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4b8cc-8eda-4c87-8ee7-78cb537909ed admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.899699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6d26d-440f-4dcd-b8fe-1ed232cd21ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.900597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6d26d-440f-4dcd-b8fe-1ed232cd21ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.900878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6d26d-440f-4dcd-b8fe-1ed232cd21ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.901154 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6d26d-440f-4dcd-b8fe-1ed232cd21ed admin admin] PATH_INFO: `/v3/projects/70624d8046024dc09b381d163ef338da/users/d23d79c1e3004a5a8fdf942b07ddd354/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.911946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4b8cc-8eda-4c87-8ee7-78cb537909ed admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:43.915487 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6d26d-440f-4dcd-b8fe-1ed232cd21ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=70624d8046024dc09b381d163ef338da, user_id=d23d79c1e3004a5a8fdf942b07ddd354, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.916901 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00913e8-2c55-46c7-8198-0aa36af54592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:43.917562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00913e8-2c55-46c7-8198-0aa36af54592 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.917877 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00913e8-2c55-46c7-8198-0aa36af54592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.918158 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00913e8-2c55-46c7-8198-0aa36af54592 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.918267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6d26d-440f-4dcd-b8fe-1ed232cd21ed admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.918844 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00913e8-2c55-46c7-8198-0aa36af54592 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:43.920759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00913e8-2c55-46c7-8198-0aa36af54592 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:43.933183 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1270/2542] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:43 2022] PUT /identity/v3/projects/70624d8046024dc09b381d163ef338da/users/d23d79c1e3004a5a8fdf942b07ddd354/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:43.934611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4434635-f9b8-4514-a811-b75ce4e21998 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:43.934896 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4434635-f9b8-4514-a811-b75ce4e21998 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:43.935172 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4434635-f9b8-4514-a811-b75ce4e21998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:43.950046 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1273/2543] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:43 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:43.950966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00913e8-2c55-46c7-8198-0aa36af54592 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:43.964681 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b4434635-f9b8-4514-a811-b75ce4e21998 None None] MFA Rules not processed for user `9a38442b69bf433fa8d3d3a7b14d49e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:43.966640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b4434635-f9b8-4514-a811-b75ce4e21998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:44.003088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:44.020005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00913e8-2c55-46c7-8198-0aa36af54592 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:44.025000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4561c9d-cab3-4d45-8c7e-0c9d1a0c8535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:44.025612 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4561c9d-cab3-4d45-8c7e-0c9d1a0c8535 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:44.026117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4561c9d-cab3-4d45-8c7e-0c9d1a0c8535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:44.026343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4561c9d-cab3-4d45-8c7e-0c9d1a0c8535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:44.027069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4561c9d-cab3-4d45-8c7e-0c9d1a0c8535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:44.028771 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4561c9d-cab3-4d45-8c7e-0c9d1a0c8535 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:44.039365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1271/2544] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:44.043175 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1274/2545] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:44.044579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4561c9d-cab3-4d45-8c7e-0c9d1a0c8535 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:44.044698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4434635-f9b8-4514-a811-b75ce4e21998 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:44.055086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4561c9d-cab3-4d45-8c7e-0c9d1a0c8535 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:44.055469 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4434635-f9b8-4514-a811-b75ce4e21998 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:44.060083 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863c8ee-7afc-4b07-a39c-3135b6f9c168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:44.060216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11794550-a0ca-4572-9b19-75b6dc039fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:44.060775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11794550-a0ca-4572-9b19-75b6dc039fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:44.060970 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863c8ee-7afc-4b07-a39c-3135b6f9c168 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:44.061072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11794550-a0ca-4572-9b19-75b6dc039fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:44.061249 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863c8ee-7afc-4b07-a39c-3135b6f9c168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:44.061360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11794550-a0ca-4572-9b19-75b6dc039fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:44.061523 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863c8ee-7afc-4b07-a39c-3135b6f9c168 admin admin] PATH_INFO: `/v3/projects/70624d8046024dc09b381d163ef338da/users/d23d79c1e3004a5a8fdf942b07ddd354/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:44.062045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11794550-a0ca-4572-9b19-75b6dc039fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:44.063592 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863c8ee-7afc-4b07-a39c-3135b6f9c168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70624d8046024dc09b381d163ef338da, user_id=d23d79c1e3004a5a8fdf942b07ddd354, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:44.063980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11794550-a0ca-4572-9b19-75b6dc039fa9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:44.066550 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863c8ee-7afc-4b07-a39c-3135b6f9c168 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:44.079287 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1275/2546] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:44 2022] PUT /identity/v3/projects/70624d8046024dc09b381d163ef338da/users/d23d79c1e3004a5a8fdf942b07ddd354/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:44.080160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863c8ee-7afc-4b07-a39c-3135b6f9c168 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:44.081876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0863c8ee-7afc-4b07-a39c-3135b6f9c168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:44.085941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1272/2547] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:44 2022] POST /identity/v3/users => generated 293 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:44.089877 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8decfa-2594-4fc0-9f9e-b33246a716ff None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:44.090165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8decfa-2594-4fc0-9f9e-b33246a716ff None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:44.090466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8decfa-2594-4fc0-9f9e-b33246a716ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:44.116550 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2c8decfa-2594-4fc0-9f9e-b33246a716ff None None] MFA Rules not processed for user `d23d79c1e3004a5a8fdf942b07ddd354`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:44.117804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2c8decfa-2594-4fc0-9f9e-b33246a716ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:44.126919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:44.144256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863c8ee-7afc-4b07-a39c-3135b6f9c168 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:44.147952 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7e38b-fa89-4990-a2d7-32847c45efa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:44.148606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7e38b-fa89-4990-a2d7-32847c45efa1 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:44.148922 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7e38b-fa89-4990-a2d7-32847c45efa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:44.149200 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7e38b-fa89-4990-a2d7-32847c45efa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:44.193257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1273/2548] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:44 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:44.197998 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8decfa-2594-4fc0-9f9e-b33246a716ff None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:44.209567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7e38b-fa89-4990-a2d7-32847c45efa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:44.211052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7e38b-fa89-4990-a2d7-32847c45efa1 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:44.221097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1276/2549] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:44.221831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7e38b-fa89-4990-a2d7-32847c45efa1 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:44.223162 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0df7e38b-fa89-4990-a2d7-32847c45efa1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:44.234347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7e38b-fa89-4990-a2d7-32847c45efa1 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:44.238841 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d3705-3791-4437-a514-6a3429ee90aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:44.239661 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d3705-3791-4437-a514-6a3429ee90aa service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:44.239936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d3705-3791-4437-a514-6a3429ee90aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:44.240244 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d3705-3791-4437-a514-6a3429ee90aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:44.250565 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d3705-3791-4437-a514-6a3429ee90aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:44.252007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d3705-3791-4437-a514-6a3429ee90aa service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:44.262400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1277/2550] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:44 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:44.271660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8decfa-2594-4fc0-9f9e-b33246a716ff None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:44.277411 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c305d2-e342-4a8f-b681-7b872eb4da57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:44.278176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c305d2-e342-4a8f-b681-7b872eb4da57 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:44.278458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c305d2-e342-4a8f-b681-7b872eb4da57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:44.278740 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c305d2-e342-4a8f-b681-7b872eb4da57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:44.279477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c305d2-e342-4a8f-b681-7b872eb4da57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:44.281085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c305d2-e342-4a8f-b681-7b872eb4da57 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:44.292817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1274/2551] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:44.297334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d3705-3791-4437-a514-6a3429ee90aa service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:44.299271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c305d2-e342-4a8f-b681-7b872eb4da57 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:44.309489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d3705-3791-4437-a514-6a3429ee90aa service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:44.310726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c305d2-e342-4a8f-b681-7b872eb4da57 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:44.314331 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3d2d9-78f8-4848-8ab0-1e54204fd58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:44.314715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea711bd2-086c-4977-ba87-df17926b3521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:44.315226 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3d2d9-78f8-4848-8ab0-1e54204fd58b admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:44.315410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea711bd2-086c-4977-ba87-df17926b3521 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:44.315630 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea711bd2-086c-4977-ba87-df17926b3521 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:44.315873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3d2d9-78f8-4848-8ab0-1e54204fd58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:44.315972 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea711bd2-086c-4977-ba87-df17926b3521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:44.316279 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3d2d9-78f8-4848-8ab0-1e54204fd58b admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:44.317155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3d2d9-78f8-4848-8ab0-1e54204fd58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:44.317739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ea711bd2-086c-4977-ba87-df17926b3521 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:44.319181 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3d2d9-78f8-4848-8ab0-1e54204fd58b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:44.331507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1278/2552] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:15:44.335751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3d2d9-78f8-4848-8ab0-1e54204fd58b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:44.346599 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3d2d9-78f8-4848-8ab0-1e54204fd58b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:44.351484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309aae9f-78cd-4e73-a6d1-4cb8d2749634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:44.352506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309aae9f-78cd-4e73-a6d1-4cb8d2749634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:44.352794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309aae9f-78cd-4e73-a6d1-4cb8d2749634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:44.353139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309aae9f-78cd-4e73-a6d1-4cb8d2749634 admin admin] PATH_INFO: `/v3/projects/adf55a22ec6c40b7bb1e293ab02ee863/users/09c51ab1c00a4eed8870b557ae5c40df/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:44.373047 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309aae9f-78cd-4e73-a6d1-4cb8d2749634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf55a22ec6c40b7bb1e293ab02ee863, user_id=09c51ab1c00a4eed8870b557ae5c40df, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:44.375746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309aae9f-78cd-4e73-a6d1-4cb8d2749634 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:44.391307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1279/2553] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:44 2022] PUT /identity/v3/projects/adf55a22ec6c40b7bb1e293ab02ee863/users/09c51ab1c00a4eed8870b557ae5c40df/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:44.397528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea711bd2-086c-4977-ba87-df17926b3521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:44.399024 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea711bd2-086c-4977-ba87-df17926b3521 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:44.401672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f6e5b-a1f5-4175-813b-dd2be00be57d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:44.402064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f6e5b-a1f5-4175-813b-dd2be00be57d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:44.402541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f6e5b-a1f5-4175-813b-dd2be00be57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:44.409655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1275/2554] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:44.433779 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-122f6e5b-a1f5-4175-813b-dd2be00be57d None None] MFA Rules not processed for user `09c51ab1c00a4eed8870b557ae5c40df`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:44.435574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-122f6e5b-a1f5-4175-813b-dd2be00be57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:44.455626 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:44.511989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1280/2555] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:44.540201 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea711bd2-086c-4977-ba87-df17926b3521 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:44.551826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea711bd2-086c-4977-ba87-df17926b3521 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:44.597831 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:44.616878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f3b2c-54db-4f0a-9214-4900bebeeee4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:44.617592 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f3b2c-54db-4f0a-9214-4900bebeeee4 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:44.617788 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f3b2c-54db-4f0a-9214-4900bebeeee4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:44.618135 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f3b2c-54db-4f0a-9214-4900bebeeee4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:44.626644 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f3b2c-54db-4f0a-9214-4900bebeeee4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:44.628131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f3b2c-54db-4f0a-9214-4900bebeeee4 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:44.638153 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1276/2556] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:45.080061 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b47c2-9b78-4119-9e81-54209dbfe514 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:45.080404 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b47c2-9b78-4119-9e81-54209dbfe514 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:45.080760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b47c2-9b78-4119-9e81-54209dbfe514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:45.102505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d59b47c2-9b78-4119-9e81-54209dbfe514 None None] MFA Rules not processed for user `2495a8406c184164982d136edc351cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:45.104118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d59b47c2-9b78-4119-9e81-54209dbfe514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:45.167950 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1281/2557] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:45.721453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f3b2c-54db-4f0a-9214-4900bebeeee4 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:45.817362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f3b2c-54db-4f0a-9214-4900bebeeee4 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:45.822105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:45.822995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:45.823339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:45.823561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] PATH_INFO: `/v3/users/9817b0c1cce549518d11fc09b57de891` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:45.824930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9817b0c1cce549518d11fc09b57de891)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:45.826907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:45.892424 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9817b0c1cce549518d11fc09b57de891'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:45.893502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9817b0c1cce549518d11fc09b57de891'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:45.904708 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9817b0c1cce549518d11fc09b57de891'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:45.922639 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9817b0c1cce549518d11fc09b57de891'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:45.923553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9817b0c1cce549518d11fc09b57de891'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:45.940543 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1277/2558] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:45 2022] DELETE /identity/v3/users/9817b0c1cce549518d11fc09b57de891 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:45.945456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ef714-bd3e-4311-bd1e-26f7f16dfb14 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:45.945946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ef714-bd3e-4311-bd1e-26f7f16dfb14 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:45.946074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ef714-bd3e-4311-bd1e-26f7f16dfb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:45.973507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b76ef714-bd3e-4311-bd1e-26f7f16dfb14 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:45.975041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b76ef714-bd3e-4311-bd1e-26f7f16dfb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:46.117570 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1282/2559] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:46.130057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:46.131718 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:46.201076 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e2713-3e70-4945-a3ca-55dc23b78f2d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:46.204711 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ef828-c609-4fa1-8646-34f7bf312b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:46.205398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ef828-c609-4fa1-8646-34f7bf312b00 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:46.205707 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ef828-c609-4fa1-8646-34f7bf312b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:46.205983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ef828-c609-4fa1-8646-34f7bf312b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:46.220384 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ef828-c609-4fa1-8646-34f7bf312b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:46.223203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ef828-c609-4fa1-8646-34f7bf312b00 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:46.229397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:46.240848 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1278/2560] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:46.523687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ef714-bd3e-4311-bd1e-26f7f16dfb14 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:46.525269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b76ef714-bd3e-4311-bd1e-26f7f16dfb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:46.537976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ef714-bd3e-4311-bd1e-26f7f16dfb14 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:46.542606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:46.543481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:46.543762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:46.544036 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] PATH_INFO: `/v3/projects/48666f7f79d249df9c54060e1bdf3839` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:46.545213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48666f7f79d249df9c54060e1bdf3839)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:46.547269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:46.596083 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48666f7f79d249df9c54060e1bdf3839', 'request_id': 'req-3dfa8352-a40b-4f65-973d-ffdd04737ab2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:46.601220 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48666f7f79d249df9c54060e1bdf3839', 'request_id': 'req-3dfa8352-a40b-4f65-973d-ffdd04737ab2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:46.603039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:46.607756 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48666f7f79d249df9c54060e1bdf3839', 'request_id': 'req-3dfa8352-a40b-4f65-973d-ffdd04737ab2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:46.621996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] The token cache is being invalidate because project 48666f7f79d249df9c54060e1bdf3839 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:46.622281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:46.622887 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:46.624419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1283/2561] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:46 2022] DELETE /identity/v3/projects/48666f7f79d249df9c54060e1bdf3839 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:46.628520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ef828-c609-4fa1-8646-34f7bf312b00 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:46.639062 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ef828-c609-4fa1-8646-34f7bf312b00 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:46.698622 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87abd2-e872-4cd2-b73d-38b1fddd7dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:46.699488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87abd2-e872-4cd2-b73d-38b1fddd7dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:46.699762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87abd2-e872-4cd2-b73d-38b1fddd7dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:46.700027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87abd2-e872-4cd2-b73d-38b1fddd7dae admin admin] PATH_INFO: `/v3/users/d23d79c1e3004a5a8fdf942b07ddd354` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:46.701260 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87abd2-e872-4cd2-b73d-38b1fddd7dae admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23d79c1e3004a5a8fdf942b07ddd354)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:46.703219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87abd2-e872-4cd2-b73d-38b1fddd7dae admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:46.753237 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9f87abd2-e872-4cd2-b73d-38b1fddd7dae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23d79c1e3004a5a8fdf942b07ddd354'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:46.753954 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9f87abd2-e872-4cd2-b73d-38b1fddd7dae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23d79c1e3004a5a8fdf942b07ddd354'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:46.762161 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9f87abd2-e872-4cd2-b73d-38b1fddd7dae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23d79c1e3004a5a8fdf942b07ddd354'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:46.774344 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9f87abd2-e872-4cd2-b73d-38b1fddd7dae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23d79c1e3004a5a8fdf942b07ddd354'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:46.774969 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9f87abd2-e872-4cd2-b73d-38b1fddd7dae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23d79c1e3004a5a8fdf942b07ddd354'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:46.786730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1279/2562] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:46 2022] DELETE /identity/v3/users/d23d79c1e3004a5a8fdf942b07ddd354 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:47.072130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:47.073783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:47.089141 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa8352-a40b-4f65-973d-ffdd04737ab2 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:47.155488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:47.156329 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:47.156608 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:47.156885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f admin admin] PATH_INFO: `/v3/projects/70624d8046024dc09b381d163ef338da` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:47.158101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=70624d8046024dc09b381d163ef338da)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:47.160088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:47.209204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70624d8046024dc09b381d163ef338da', 'request_id': 'req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:47.215256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70624d8046024dc09b381d163ef338da', 'request_id': 'req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:47.222529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70624d8046024dc09b381d163ef338da', 'request_id': 'req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:47.235712 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f admin admin] The token cache is being invalidate because project 70624d8046024dc09b381d163ef338da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:47.236006 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:47.236637 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cd5d257b-50b4-4ff9-9c38-c3db3f6d9e0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:47.238406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1284/2563] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:47 2022] DELETE /identity/v3/projects/70624d8046024dc09b381d163ef338da => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:47.243560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a18f77-cfdc-473b-8702-3d8e8b81cf67 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:47.243859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a18f77-cfdc-473b-8702-3d8e8b81cf67 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:47.244157 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a18f77-cfdc-473b-8702-3d8e8b81cf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:47.262305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-89a18f77-cfdc-473b-8702-3d8e8b81cf67 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:47.263562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-89a18f77-cfdc-473b-8702-3d8e8b81cf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:47.333717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1280/2564] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:47.341878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531489dc-fe6b-461b-bc95-b721c9c318f1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:47.342182 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531489dc-fe6b-461b-bc95-b721c9c318f1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:47.342445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531489dc-fe6b-461b-bc95-b721c9c318f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:47.362716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-531489dc-fe6b-461b-bc95-b721c9c318f1 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:47.363936 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-531489dc-fe6b-461b-bc95-b721c9c318f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:47.372070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1285/2565] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:47.376291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a18f77-cfdc-473b-8702-3d8e8b81cf67 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:47.386566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a18f77-cfdc-473b-8702-3d8e8b81cf67 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:47.391165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db186655-e80a-4f6c-9b49-16586f24049c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:47.391806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db186655-e80a-4f6c-9b49-16586f24049c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:47.392096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db186655-e80a-4f6c-9b49-16586f24049c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:47.392376 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db186655-e80a-4f6c-9b49-16586f24049c admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:47.393761 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db186655-e80a-4f6c-9b49-16586f24049c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:15:47.394080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db186655-e80a-4f6c-9b49-16586f24049c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:47.395709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db186655-e80a-4f6c-9b49-16586f24049c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:47.407178 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1281/2566] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:15:47.412062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531489dc-fe6b-461b-bc95-b721c9c318f1 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:47.421338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531489dc-fe6b-461b-bc95-b721c9c318f1 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:47.427098 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62333b58-bbf8-433f-93c6-6dd55a7a1624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:47.427765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62333b58-bbf8-433f-93c6-6dd55a7a1624 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:47.428050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62333b58-bbf8-433f-93c6-6dd55a7a1624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:47.428316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62333b58-bbf8-433f-93c6-6dd55a7a1624 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:47.429062 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62333b58-bbf8-433f-93c6-6dd55a7a1624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:47.431080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62333b58-bbf8-433f-93c6-6dd55a7a1624 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:47.460022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1286/2567] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:47 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:47.464670 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db186655-e80a-4f6c-9b49-16586f24049c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:47.475491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db186655-e80a-4f6c-9b49-16586f24049c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:47.479366 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:47.535253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df92da-8316-43b9-875d-a68de94ae031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:47.535929 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df92da-8316-43b9-875d-a68de94ae031 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:47.536214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df92da-8316-43b9-875d-a68de94ae031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:47.536518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df92da-8316-43b9-875d-a68de94ae031 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:47.537206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df92da-8316-43b9-875d-a68de94ae031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:47.539173 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df92da-8316-43b9-875d-a68de94ae031 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:47.564554 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1282/2568] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:47 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:47.568952 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62333b58-bbf8-433f-93c6-6dd55a7a1624 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:47.580069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62333b58-bbf8-433f-93c6-6dd55a7a1624 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:47.585886 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834bc1b-946c-4539-bea9-9e044d45c25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:47.586540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834bc1b-946c-4539-bea9-9e044d45c25e admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:47.586819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834bc1b-946c-4539-bea9-9e044d45c25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:47.587087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834bc1b-946c-4539-bea9-9e044d45c25e admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:47.588138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834bc1b-946c-4539-bea9-9e044d45c25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:47.589769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834bc1b-946c-4539-bea9-9e044d45c25e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:47.600700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1287/2569] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:47.604908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df92da-8316-43b9-875d-a68de94ae031 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:47.615786 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df92da-8316-43b9-875d-a68de94ae031 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:47.621052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1eedc7-19c7-45b9-ac48-054f43c00d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:47.621767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eedc7-19c7-45b9-ac48-054f43c00d5e admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:47.622081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eedc7-19c7-45b9-ac48-054f43c00d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:47.622357 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eedc7-19c7-45b9-ac48-054f43c00d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:47.623084 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1eedc7-19c7-45b9-ac48-054f43c00d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:47.624703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1eedc7-19c7-45b9-ac48-054f43c00d5e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:47.635065 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1283/2570] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:47.639515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834bc1b-946c-4539-bea9-9e044d45c25e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:47.650900 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834bc1b-946c-4539-bea9-9e044d45c25e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:47.656967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba008c-da49-49fb-ba4b-c9cc4724dc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:47.657872 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba008c-da49-49fb-ba4b-c9cc4724dc14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:47.658189 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba008c-da49-49fb-ba4b-c9cc4724dc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:47.658527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba008c-da49-49fb-ba4b-c9cc4724dc14 admin admin] PATH_INFO: `/v3/projects/53c074761b1e487193ecd57d170fcaad/users/12108c5e9e36477fb14396228546e981/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:47.673923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba008c-da49-49fb-ba4b-c9cc4724dc14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c074761b1e487193ecd57d170fcaad, user_id=12108c5e9e36477fb14396228546e981, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:47.676641 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba008c-da49-49fb-ba4b-c9cc4724dc14 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:47.691854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1288/2571] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:47 2022] PUT /identity/v3/projects/53c074761b1e487193ecd57d170fcaad/users/12108c5e9e36477fb14396228546e981/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:47.704460 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e46970-e8b7-44b2-8d73-a4dcb99ac261 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:47.704830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e46970-e8b7-44b2-8d73-a4dcb99ac261 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:47.705240 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e46970-e8b7-44b2-8d73-a4dcb99ac261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:47.738085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-88e46970-e8b7-44b2-8d73-a4dcb99ac261 None None] MFA Rules not processed for user `12108c5e9e36477fb14396228546e981`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:47.739312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-88e46970-e8b7-44b2-8d73-a4dcb99ac261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:47.805913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1284/2572] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:47.810707 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba008c-da49-49fb-ba4b-c9cc4724dc14 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:47.829491 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:47.878266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba008c-da49-49fb-ba4b-c9cc4724dc14 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:47.883374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6591dfd-7187-435f-9091-730b19c2c1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:47.884037 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6591dfd-7187-435f-9091-730b19c2c1ec admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:47.884316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6591dfd-7187-435f-9091-730b19c2c1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:47.884614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6591dfd-7187-435f-9091-730b19c2c1ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:47.885324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6591dfd-7187-435f-9091-730b19c2c1ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:47.887336 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6591dfd-7187-435f-9091-730b19c2c1ec admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:47.919749 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1289/2573] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:47 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:47.921561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e46970-e8b7-44b2-8d73-a4dcb99ac261 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:47.932996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e46970-e8b7-44b2-8d73-a4dcb99ac261 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:47.996206 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8503624-3ddf-4e1a-a61e-bdbb0ca3aca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:47.996886 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8503624-3ddf-4e1a-a61e-bdbb0ca3aca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:47.997188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8503624-3ddf-4e1a-a61e-bdbb0ca3aca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:47.997457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8503624-3ddf-4e1a-a61e-bdbb0ca3aca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:47.998231 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8503624-3ddf-4e1a-a61e-bdbb0ca3aca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:48.000159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8503624-3ddf-4e1a-a61e-bdbb0ca3aca6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:48.024535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1285/2574] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:47 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:48.025764 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6591dfd-7187-435f-9091-730b19c2c1ec admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:48.036401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6591dfd-7187-435f-9091-730b19c2c1ec admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:48.042030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ea47c-58fc-42ea-81f4-59400ac4ae5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:48.042700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ea47c-58fc-42ea-81f4-59400ac4ae5d admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:48.042985 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ea47c-58fc-42ea-81f4-59400ac4ae5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:48.043270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ea47c-58fc-42ea-81f4-59400ac4ae5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:48.044008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ea47c-58fc-42ea-81f4-59400ac4ae5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:48.045655 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ea47c-58fc-42ea-81f4-59400ac4ae5d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:48.056630 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1290/2575] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:15:48.060842 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8503624-3ddf-4e1a-a61e-bdbb0ca3aca6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:48.067600 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:48.071171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8503624-3ddf-4e1a-a61e-bdbb0ca3aca6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:48.076960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf4ad9-a4bd-4928-8f61-00615d261efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:48.077900 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf4ad9-a4bd-4928-8f61-00615d261efa admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:48.078188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf4ad9-a4bd-4928-8f61-00615d261efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:48.078472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf4ad9-a4bd-4928-8f61-00615d261efa admin admin] PATH_INFO: `/v3/projects/5a97bd00f8eb43198ec6f395404176d3/users/0babc37a166d42cca8abad92a2c3fee5/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:48.095459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf4ad9-a4bd-4928-8f61-00615d261efa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a97bd00f8eb43198ec6f395404176d3, user_id=0babc37a166d42cca8abad92a2c3fee5, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:48.098401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf4ad9-a4bd-4928-8f61-00615d261efa admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:48.113974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1286/2576] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:48 2022] PUT /identity/v3/projects/5a97bd00f8eb43198ec6f395404176d3/users/0babc37a166d42cca8abad92a2c3fee5/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:48.118140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ea47c-58fc-42ea-81f4-59400ac4ae5d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:48.129094 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ea47c-58fc-42ea-81f4-59400ac4ae5d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:48.188783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828cb28-bff1-4c8c-94e8-b7f2bfecaee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:48.189351 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828cb28-bff1-4c8c-94e8-b7f2bfecaee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:48.189649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828cb28-bff1-4c8c-94e8-b7f2bfecaee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:48.189949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828cb28-bff1-4c8c-94e8-b7f2bfecaee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:48.190713 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828cb28-bff1-4c8c-94e8-b7f2bfecaee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:48.192505 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828cb28-bff1-4c8c-94e8-b7f2bfecaee7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:48.202199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1291/2577] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:48.207113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf4ad9-a4bd-4928-8f61-00615d261efa admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:48.218664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf4ad9-a4bd-4928-8f61-00615d261efa admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:48.224937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f537db-6d04-40e0-9562-a5cd165794ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:48.225876 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f537db-6d04-40e0-9562-a5cd165794ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:48.226156 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f537db-6d04-40e0-9562-a5cd165794ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:48.226440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f537db-6d04-40e0-9562-a5cd165794ec admin admin] PATH_INFO: `/v3/projects/5a97bd00f8eb43198ec6f395404176d3/users/0babc37a166d42cca8abad92a2c3fee5/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:48.228649 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f537db-6d04-40e0-9562-a5cd165794ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a97bd00f8eb43198ec6f395404176d3, user_id=0babc37a166d42cca8abad92a2c3fee5, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:48.231375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f537db-6d04-40e0-9562-a5cd165794ec admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:48.242694 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1287/2578] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:48 2022] PUT /identity/v3/projects/5a97bd00f8eb43198ec6f395404176d3/users/0babc37a166d42cca8abad92a2c3fee5/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:48.258342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4109529-a448-49fc-9e8d-1d9a838cc2cb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:48.258770 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4109529-a448-49fc-9e8d-1d9a838cc2cb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:48.259190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4109529-a448-49fc-9e8d-1d9a838cc2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:48.295710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f4109529-a448-49fc-9e8d-1d9a838cc2cb None None] MFA Rules not processed for user `0babc37a166d42cca8abad92a2c3fee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:48.297498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f4109529-a448-49fc-9e8d-1d9a838cc2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:48.372009 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1292/2579] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:48.384209 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f537db-6d04-40e0-9562-a5cd165794ec admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:48.386410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-42f537db-6d04-40e0-9562-a5cd165794ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:48.445437 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f537db-6d04-40e0-9562-a5cd165794ec admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:48.449397 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74a278-647e-4021-8847-4457d989a014 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:48.450077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74a278-647e-4021-8847-4457d989a014 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:48.450361 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74a278-647e-4021-8847-4457d989a014 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:48.450636 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74a278-647e-4021-8847-4457d989a014 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:48.460002 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74a278-647e-4021-8847-4457d989a014 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:48.461454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74a278-647e-4021-8847-4457d989a014 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:48.471257 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1288/2580] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:48.835312 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4109529-a448-49fc-9e8d-1d9a838cc2cb None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:48.848517 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:48.910130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4109529-a448-49fc-9e8d-1d9a838cc2cb None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:48.915406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:48.916319 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:48.916585 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:48.916875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] PATH_INFO: `/v3/users/12108c5e9e36477fb14396228546e981` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:48.918113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12108c5e9e36477fb14396228546e981)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:48.920069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:48.981134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12108c5e9e36477fb14396228546e981'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:48.981816 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12108c5e9e36477fb14396228546e981'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:48.991498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12108c5e9e36477fb14396228546e981'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:49.004484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12108c5e9e36477fb14396228546e981'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:49.005165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12108c5e9e36477fb14396228546e981'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:49.018568 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1293/2581] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:48 2022] DELETE /identity/v3/users/12108c5e9e36477fb14396228546e981 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:49.026175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8da667-153b-4657-be8a-eb83fa039a10 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:49.026175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8da667-153b-4657-be8a-eb83fa039a10 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:49.026175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8da667-153b-4657-be8a-eb83fa039a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:49.044389 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3e8da667-153b-4657-be8a-eb83fa039a10 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:49.045783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3e8da667-153b-4657-be8a-eb83fa039a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:49.108343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1289/2582] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:49.119381 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:49.120872 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:49.186663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f16ac0-249c-4d78-b587-b0ae86cc1aa5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:49.190670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f580a61-d1d0-414f-bd9a-9348adf10b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:49.191394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f580a61-d1d0-414f-bd9a-9348adf10b54 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:49.191793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f580a61-d1d0-414f-bd9a-9348adf10b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:49.192073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f580a61-d1d0-414f-bd9a-9348adf10b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:49.203667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f580a61-d1d0-414f-bd9a-9348adf10b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:49.205240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f580a61-d1d0-414f-bd9a-9348adf10b54 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:49.215803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1294/2583] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:49.692737 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8da667-153b-4657-be8a-eb83fa039a10 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:49.694410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3e8da667-153b-4657-be8a-eb83fa039a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:49.713361 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8da667-153b-4657-be8a-eb83fa039a10 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:49.719838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:49.726188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:49.726188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:49.726188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] PATH_INFO: `/v3/projects/53c074761b1e487193ecd57d170fcaad` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:49.730838 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=53c074761b1e487193ecd57d170fcaad)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:49.733513 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:49.795439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53c074761b1e487193ecd57d170fcaad', 'request_id': 'req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:49.802335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53c074761b1e487193ecd57d170fcaad', 'request_id': 'req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:49.810032 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53c074761b1e487193ecd57d170fcaad', 'request_id': 'req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:49.815013 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:49.828375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] The token cache is being invalidate because project 53c074761b1e487193ecd57d170fcaad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:49.828860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:49.829978 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:49.835362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1290/2584] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:49 2022] DELETE /identity/v3/projects/53c074761b1e487193ecd57d170fcaad => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:49.840561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f580a61-d1d0-414f-bd9a-9348adf10b54 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:49.856493 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f580a61-d1d0-414f-bd9a-9348adf10b54 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:49.893978 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:49.923169 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490eec44-2530-49df-a0b2-ea55509622ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:49.924130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490eec44-2530-49df-a0b2-ea55509622ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:49.924503 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490eec44-2530-49df-a0b2-ea55509622ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:49.924865 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490eec44-2530-49df-a0b2-ea55509622ca admin admin] PATH_INFO: `/v3/users/0babc37a166d42cca8abad92a2c3fee5` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:49.926401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490eec44-2530-49df-a0b2-ea55509622ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=0babc37a166d42cca8abad92a2c3fee5)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:49.928418 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490eec44-2530-49df-a0b2-ea55509622ca admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:49.994960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-490eec44-2530-49df-a0b2-ea55509622ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0babc37a166d42cca8abad92a2c3fee5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:49.995868 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-490eec44-2530-49df-a0b2-ea55509622ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0babc37a166d42cca8abad92a2c3fee5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:50.009800 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-490eec44-2530-49df-a0b2-ea55509622ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0babc37a166d42cca8abad92a2c3fee5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:50.027736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-490eec44-2530-49df-a0b2-ea55509622ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0babc37a166d42cca8abad92a2c3fee5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:50.028547 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-490eec44-2530-49df-a0b2-ea55509622ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0babc37a166d42cca8abad92a2c3fee5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:50.047677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1295/2585] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:49 2022] DELETE /identity/v3/users/0babc37a166d42cca8abad92a2c3fee5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:50.340003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:50.341904 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:50.357226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04c05d-f2f3-4681-b1be-f2fb29e98e1a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:50.420255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:50.421131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:50.421423 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:50.421731 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237 admin admin] PATH_INFO: `/v3/projects/5a97bd00f8eb43198ec6f395404176d3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:50.423209 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a97bd00f8eb43198ec6f395404176d3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:50.425217 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:50.476038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a97bd00f8eb43198ec6f395404176d3', 'request_id': 'req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:50.482370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a97bd00f8eb43198ec6f395404176d3', 'request_id': 'req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:50.488491 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a97bd00f8eb43198ec6f395404176d3', 'request_id': 'req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:50.503236 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237 admin admin] The token cache is being invalidate because project 5a97bd00f8eb43198ec6f395404176d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:50.503520 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:50.504149 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-37ad04bd-e3b9-41b7-8b9f-ffb85491d237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:50.505764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1291/2586] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:50 2022] DELETE /identity/v3/projects/5a97bd00f8eb43198ec6f395404176d3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:50.510791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ed4fb-b647-45d1-bdc9-f8a5b7976910 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:50.511273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ed4fb-b647-45d1-bdc9-f8a5b7976910 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:50.511273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ed4fb-b647-45d1-bdc9-f8a5b7976910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:50.529506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a59ed4fb-b647-45d1-bdc9-f8a5b7976910 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:50.530746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a59ed4fb-b647-45d1-bdc9-f8a5b7976910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:50.592582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1296/2587] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:50.601598 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b8ecb-6c5c-47ed-b2d2-62bb7ea93ded None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:50.601920 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b8ecb-6c5c-47ed-b2d2-62bb7ea93ded None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:50.602207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b8ecb-6c5c-47ed-b2d2-62bb7ea93ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:50.621061 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e76b8ecb-6c5c-47ed-b2d2-62bb7ea93ded None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:50.622299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e76b8ecb-6c5c-47ed-b2d2-62bb7ea93ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:50.631111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1292/2588] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:50.636437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ed4fb-b647-45d1-bdc9-f8a5b7976910 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:50.646481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ed4fb-b647-45d1-bdc9-f8a5b7976910 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:50.651693 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf79f6-2009-4aa0-a89f-478f102cc3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:50.652352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf79f6-2009-4aa0-a89f-478f102cc3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:50.652636 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf79f6-2009-4aa0-a89f-478f102cc3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:50.652884 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf79f6-2009-4aa0-a89f-478f102cc3c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:50.653654 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf79f6-2009-4aa0-a89f-478f102cc3c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:15:50.653980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf79f6-2009-4aa0-a89f-478f102cc3c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:50.655626 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf79f6-2009-4aa0-a89f-478f102cc3c2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:50.668335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1297/2589] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:15:50.674690 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b8ecb-6c5c-47ed-b2d2-62bb7ea93ded None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:50.684503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b8ecb-6c5c-47ed-b2d2-62bb7ea93ded None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:50.690371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64324a9-adf1-44a5-ba21-df83d9bec1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:50.691464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64324a9-adf1-44a5-ba21-df83d9bec1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:50.691668 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64324a9-adf1-44a5-ba21-df83d9bec1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:50.691943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64324a9-adf1-44a5-ba21-df83d9bec1a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:50.692638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64324a9-adf1-44a5-ba21-df83d9bec1a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:50.694732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64324a9-adf1-44a5-ba21-df83d9bec1a7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:50.727659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1293/2590] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:50 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:50.731064 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf79f6-2009-4aa0-a89f-478f102cc3c2 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:50.741835 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf79f6-2009-4aa0-a89f-478f102cc3c2 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:50.779131 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:50.798556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa21e1-e5cb-43e2-8e22-12470d54ce13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:50.799228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa21e1-e5cb-43e2-8e22-12470d54ce13 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:50.799507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa21e1-e5cb-43e2-8e22-12470d54ce13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:50.799790 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa21e1-e5cb-43e2-8e22-12470d54ce13 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:50.800875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa21e1-e5cb-43e2-8e22-12470d54ce13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:50.802852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa21e1-e5cb-43e2-8e22-12470d54ce13 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:50.836515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1298/2591] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:50 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:50.836796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64324a9-adf1-44a5-ba21-df83d9bec1a7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:50.843755 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64324a9-adf1-44a5-ba21-df83d9bec1a7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:50.849433 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11b06d-0d46-49ae-aee3-d4599878e30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:50.850274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11b06d-0d46-49ae-aee3-d4599878e30c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:50.850610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11b06d-0d46-49ae-aee3-d4599878e30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:50.850944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11b06d-0d46-49ae-aee3-d4599878e30c admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:50.851790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11b06d-0d46-49ae-aee3-d4599878e30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:50.853515 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11b06d-0d46-49ae-aee3-d4599878e30c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:50.866865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1294/2592] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:50.890653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa21e1-e5cb-43e2-8e22-12470d54ce13 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:50.901395 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa21e1-e5cb-43e2-8e22-12470d54ce13 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:50.906656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f977be-d825-4231-862c-42e593cbb5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:50.907320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f977be-d825-4231-862c-42e593cbb5ca admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:50.907604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f977be-d825-4231-862c-42e593cbb5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:50.907879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f977be-d825-4231-862c-42e593cbb5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:50.908625 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f977be-d825-4231-862c-42e593cbb5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:50.910848 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f977be-d825-4231-862c-42e593cbb5ca admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:50.921411 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1299/2593] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:50.926210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11b06d-0d46-49ae-aee3-d4599878e30c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:50.941920 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11b06d-0d46-49ae-aee3-d4599878e30c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:50.948053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c74ad-8808-4608-a50e-a204fc5249a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:50.949027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c74ad-8808-4608-a50e-a204fc5249a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:50.949393 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c74ad-8808-4608-a50e-a204fc5249a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:50.949804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c74ad-8808-4608-a50e-a204fc5249a9 admin admin] PATH_INFO: `/v3/projects/1e9987751b85454997daa4f8ec41822e/users/4249c1e8b9be4c3a9ebcf0144081d5e4/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:50.966801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c74ad-8808-4608-a50e-a204fc5249a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9987751b85454997daa4f8ec41822e, user_id=4249c1e8b9be4c3a9ebcf0144081d5e4, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:50.969507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c74ad-8808-4608-a50e-a204fc5249a9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:50.985050 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1295/2594] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:50 2022] PUT /identity/v3/projects/1e9987751b85454997daa4f8ec41822e/users/4249c1e8b9be4c3a9ebcf0144081d5e4/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:50.995707 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2919b-b792-4e05-8ba7-0840b808f022 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:50.995999 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2919b-b792-4e05-8ba7-0840b808f022 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:50.996307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2919b-b792-4e05-8ba7-0840b808f022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:51.025678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-eca2919b-b792-4e05-8ba7-0840b808f022 None None] MFA Rules not processed for user `4249c1e8b9be4c3a9ebcf0144081d5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:51.026914 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-eca2919b-b792-4e05-8ba7-0840b808f022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:51.096336 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1300/2595] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:51.101213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c74ad-8808-4608-a50e-a204fc5249a9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:51.138523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:51.179574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c74ad-8808-4608-a50e-a204fc5249a9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:51.184377 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e996e36-2a35-4df0-a121-920be372e1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:51.185060 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e996e36-2a35-4df0-a121-920be372e1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:51.185345 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e996e36-2a35-4df0-a121-920be372e1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:51.185643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e996e36-2a35-4df0-a121-920be372e1f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:51.186365 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e996e36-2a35-4df0-a121-920be372e1f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:51.188354 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e996e36-2a35-4df0-a121-920be372e1f4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:51.216295 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1296/2596] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:51 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:51.220709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2919b-b792-4e05-8ba7-0840b808f022 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:51.232098 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2919b-b792-4e05-8ba7-0840b808f022 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:51.293724 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18e868-b277-4779-b02d-0bb82802b207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:51.294394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18e868-b277-4779-b02d-0bb82802b207 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:51.294681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18e868-b277-4779-b02d-0bb82802b207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:51.294961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18e868-b277-4779-b02d-0bb82802b207 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:51.295699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18e868-b277-4779-b02d-0bb82802b207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:51.297648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18e868-b277-4779-b02d-0bb82802b207 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:51.323729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1301/2597] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:51 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:51.325260 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e996e36-2a35-4df0-a121-920be372e1f4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:51.338532 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e996e36-2a35-4df0-a121-920be372e1f4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:51.343654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323b620-76e2-430b-9d29-a14c7fbb7bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:51.344361 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323b620-76e2-430b-9d29-a14c7fbb7bad admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:51.344647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323b620-76e2-430b-9d29-a14c7fbb7bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:51.344923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323b620-76e2-430b-9d29-a14c7fbb7bad admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:51.345699 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323b620-76e2-430b-9d29-a14c7fbb7bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:51.348006 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323b620-76e2-430b-9d29-a14c7fbb7bad admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:51.359192 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1297/2598] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:15:51.363847 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18e868-b277-4779-b02d-0bb82802b207 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:51.375168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18e868-b277-4779-b02d-0bb82802b207 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:51.380683 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a6bad-bbcb-43b6-a28f-f851961bdf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:51.381586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a6bad-bbcb-43b6-a28f-f851961bdf5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:51.381895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a6bad-bbcb-43b6-a28f-f851961bdf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:51.382188 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a6bad-bbcb-43b6-a28f-f851961bdf5b admin admin] PATH_INFO: `/v3/projects/802d6ad68d69443e84094a1413088cfe/users/3471828456eb47fc999973e989502cfa/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:51.398656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a6bad-bbcb-43b6-a28f-f851961bdf5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=802d6ad68d69443e84094a1413088cfe, user_id=3471828456eb47fc999973e989502cfa, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:51.401513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a6bad-bbcb-43b6-a28f-f851961bdf5b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:51.416035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1302/2599] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:51 2022] PUT /identity/v3/projects/802d6ad68d69443e84094a1413088cfe/users/3471828456eb47fc999973e989502cfa/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:51.420141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323b620-76e2-430b-9d29-a14c7fbb7bad admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:51.431915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323b620-76e2-430b-9d29-a14c7fbb7bad admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:51.495409 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad121d4-1840-47a2-b31f-80ff36cf637d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:51.496233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad121d4-1840-47a2-b31f-80ff36cf637d admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:51.496581 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad121d4-1840-47a2-b31f-80ff36cf637d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:51.496924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad121d4-1840-47a2-b31f-80ff36cf637d admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:51.497706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad121d4-1840-47a2-b31f-80ff36cf637d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:51.499368 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad121d4-1840-47a2-b31f-80ff36cf637d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:51.512821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1298/2600] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:51.514107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a6bad-bbcb-43b6-a28f-f851961bdf5b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:51.525923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a6bad-bbcb-43b6-a28f-f851961bdf5b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:51.531392 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42017700-9251-4593-8c52-3123a41eeb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:51.532130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42017700-9251-4593-8c52-3123a41eeb31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:51.532410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42017700-9251-4593-8c52-3123a41eeb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:51.533462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42017700-9251-4593-8c52-3123a41eeb31 admin admin] PATH_INFO: `/v3/projects/802d6ad68d69443e84094a1413088cfe/users/3471828456eb47fc999973e989502cfa/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:51.535736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42017700-9251-4593-8c52-3123a41eeb31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=802d6ad68d69443e84094a1413088cfe, user_id=3471828456eb47fc999973e989502cfa, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:51.538679 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42017700-9251-4593-8c52-3123a41eeb31 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:51.546157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:51.551607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1303/2601] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:51 2022] PUT /identity/v3/projects/802d6ad68d69443e84094a1413088cfe/users/3471828456eb47fc999973e989502cfa/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:51.562179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821d90c-fb6a-478a-93ed-aee675489a04 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:51.562477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821d90c-fb6a-478a-93ed-aee675489a04 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:51.562764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821d90c-fb6a-478a-93ed-aee675489a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:51.590383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9821d90c-fb6a-478a-93ed-aee675489a04 None None] MFA Rules not processed for user `3471828456eb47fc999973e989502cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:51.591613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9821d90c-fb6a-478a-93ed-aee675489a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:51.663996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1299/2602] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:51.675765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42017700-9251-4593-8c52-3123a41eeb31 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:51.677507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-42017700-9251-4593-8c52-3123a41eeb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:51.737460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42017700-9251-4593-8c52-3123a41eeb31 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:51.741451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f97dd-48ea-4118-98c6-4bff7511530d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:51.742142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f97dd-48ea-4118-98c6-4bff7511530d service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:51.742424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f97dd-48ea-4118-98c6-4bff7511530d service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:51.742699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f97dd-48ea-4118-98c6-4bff7511530d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:51.752156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f97dd-48ea-4118-98c6-4bff7511530d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:51.754010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f97dd-48ea-4118-98c6-4bff7511530d service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:51.764980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1304/2603] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:51.838633 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821d90c-fb6a-478a-93ed-aee675489a04 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:51.852678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821d90c-fb6a-478a-93ed-aee675489a04 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:51.856474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56507d6-9c15-4f7e-aa83-e4438994a1d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:51.857094 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56507d6-9c15-4f7e-aa83-e4438994a1d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:51.857382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56507d6-9c15-4f7e-aa83-e4438994a1d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:51.857716 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56507d6-9c15-4f7e-aa83-e4438994a1d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:51.871508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:51.918313 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56507d6-9c15-4f7e-aa83-e4438994a1d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:51.919750 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56507d6-9c15-4f7e-aa83-e4438994a1d9 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:51.928996 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1300/2604] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:52.222710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f97dd-48ea-4118-98c6-4bff7511530d service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:52.292154 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f97dd-48ea-4118-98c6-4bff7511530d service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:52.296896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:52.297867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:52.298157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:52.298429 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] PATH_INFO: `/v3/users/4249c1e8b9be4c3a9ebcf0144081d5e4` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:52.299812 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4249c1e8b9be4c3a9ebcf0144081d5e4)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:52.302383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:52.357674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4249c1e8b9be4c3a9ebcf0144081d5e4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:52.358335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4249c1e8b9be4c3a9ebcf0144081d5e4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:52.367451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4249c1e8b9be4c3a9ebcf0144081d5e4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:52.380561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4249c1e8b9be4c3a9ebcf0144081d5e4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:52.381284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4249c1e8b9be4c3a9ebcf0144081d5e4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:52.394155 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1305/2605] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:52 2022] DELETE /identity/v3/users/4249c1e8b9be4c3a9ebcf0144081d5e4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:52.398847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf45991-eec7-4b9a-a2c8-4aa3c812cbba None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:52.399337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf45991-eec7-4b9a-a2c8-4aa3c812cbba None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:52.399454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf45991-eec7-4b9a-a2c8-4aa3c812cbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:52.419402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-caf45991-eec7-4b9a-a2c8-4aa3c812cbba None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:52.420109 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-caf45991-eec7-4b9a-a2c8-4aa3c812cbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:52.492060 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1301/2606] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:15:52.501824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:52.503563 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:52.542034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:52.595912 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c45a5a-ec13-48c5-8a91-18eb4f92dd17 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:52.600808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c0072-46fd-42fd-b191-77f2b790e6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:52.601866 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c0072-46fd-42fd-b191-77f2b790e6ce service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:52.602387 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c0072-46fd-42fd-b191-77f2b790e6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:52.602976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c0072-46fd-42fd-b191-77f2b790e6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:52.633337 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c0072-46fd-42fd-b191-77f2b790e6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:52.635701 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c0072-46fd-42fd-b191-77f2b790e6ce service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:52.662538 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1306/2607] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:52.978289 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf45991-eec7-4b9a-a2c8-4aa3c812cbba None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:52.979944 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-caf45991-eec7-4b9a-a2c8-4aa3c812cbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:53.000251 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf45991-eec7-4b9a-a2c8-4aa3c812cbba None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:53.014932 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:53.016140 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:53.016630 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:53.017516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] PATH_INFO: `/v3/projects/1e9987751b85454997daa4f8ec41822e` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:53.019011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9987751b85454997daa4f8ec41822e)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:53.022042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:53.089784 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9987751b85454997daa4f8ec41822e', 'request_id': 'req-436221cc-c90c-410b-b4ff-493b51e1c66d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.097152 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9987751b85454997daa4f8ec41822e', 'request_id': 'req-436221cc-c90c-410b-b4ff-493b51e1c66d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.103855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9987751b85454997daa4f8ec41822e', 'request_id': 'req-436221cc-c90c-410b-b4ff-493b51e1c66d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.121069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] The token cache is being invalidate because project 1e9987751b85454997daa4f8ec41822e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:53.121069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.122979 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.125480 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1302/2608] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:52 2022] DELETE /identity/v3/projects/1e9987751b85454997daa4f8ec41822e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:53.129528 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c0072-46fd-42fd-b191-77f2b790e6ce service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:53.147193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c0072-46fd-42fd-b191-77f2b790e6ce service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:53.219143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c867d4-e52a-4285-9c44-e1518d6dff33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:53.220044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c867d4-e52a-4285-9c44-e1518d6dff33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:53.220328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c867d4-e52a-4285-9c44-e1518d6dff33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:53.220606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c867d4-e52a-4285-9c44-e1518d6dff33 admin admin] PATH_INFO: `/v3/users/3471828456eb47fc999973e989502cfa` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:53.222593 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c867d4-e52a-4285-9c44-e1518d6dff33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3471828456eb47fc999973e989502cfa)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:53.224693 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c867d4-e52a-4285-9c44-e1518d6dff33 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:53.281207 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-30c867d4-e52a-4285-9c44-e1518d6dff33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3471828456eb47fc999973e989502cfa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.282171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-30c867d4-e52a-4285-9c44-e1518d6dff33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3471828456eb47fc999973e989502cfa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.294606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-30c867d4-e52a-4285-9c44-e1518d6dff33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3471828456eb47fc999973e989502cfa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.318809 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-30c867d4-e52a-4285-9c44-e1518d6dff33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3471828456eb47fc999973e989502cfa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.319744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-30c867d4-e52a-4285-9c44-e1518d6dff33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3471828456eb47fc999973e989502cfa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.333345 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1307/2609] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:53 2022] DELETE /identity/v3/users/3471828456eb47fc999973e989502cfa => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:53.620335 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:53.621903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:53.635808 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436221cc-c90c-410b-b4ff-493b51e1c66d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:53.674371 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:53.700030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591746bb-3cc1-40ea-9346-a2789b585b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:53.700830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591746bb-3cc1-40ea-9346-a2789b585b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:53.701114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591746bb-3cc1-40ea-9346-a2789b585b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:53.701391 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591746bb-3cc1-40ea-9346-a2789b585b94 admin admin] PATH_INFO: `/v3/projects/802d6ad68d69443e84094a1413088cfe` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:53.702619 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591746bb-3cc1-40ea-9346-a2789b585b94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=802d6ad68d69443e84094a1413088cfe)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:53.704544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591746bb-3cc1-40ea-9346-a2789b585b94 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:53.754870 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-591746bb-3cc1-40ea-9346-a2789b585b94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '802d6ad68d69443e84094a1413088cfe', 'request_id': 'req-591746bb-3cc1-40ea-9346-a2789b585b94'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.761091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-591746bb-3cc1-40ea-9346-a2789b585b94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '802d6ad68d69443e84094a1413088cfe', 'request_id': 'req-591746bb-3cc1-40ea-9346-a2789b585b94'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.767194 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-591746bb-3cc1-40ea-9346-a2789b585b94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '802d6ad68d69443e84094a1413088cfe', 'request_id': 'req-591746bb-3cc1-40ea-9346-a2789b585b94'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.782886 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-591746bb-3cc1-40ea-9346-a2789b585b94 admin admin] The token cache is being invalidate because project 802d6ad68d69443e84094a1413088cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:53.783173 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-591746bb-3cc1-40ea-9346-a2789b585b94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.783909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-591746bb-3cc1-40ea-9346-a2789b585b94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:53.785726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1303/2610] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:53 2022] DELETE /identity/v3/projects/802d6ad68d69443e84094a1413088cfe => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:53.790807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37cb32-208d-480a-8438-37243d2ffdf2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:53.791302 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37cb32-208d-480a-8438-37243d2ffdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:53.791425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37cb32-208d-480a-8438-37243d2ffdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:53.809542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5d37cb32-208d-480a-8438-37243d2ffdf2 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:53.810794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5d37cb32-208d-480a-8438-37243d2ffdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:53.853653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:53.876416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1308/2611] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:53.886288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe426c-b432-416a-b95d-ba7c92ad2604 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:53.886579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe426c-b432-416a-b95d-ba7c92ad2604 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:53.886860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe426c-b432-416a-b95d-ba7c92ad2604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:53.905864 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-39fe426c-b432-416a-b95d-ba7c92ad2604 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:53.907151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-39fe426c-b432-416a-b95d-ba7c92ad2604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:53.915916 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1304/2612] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:53.920298 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37cb32-208d-480a-8438-37243d2ffdf2 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:53.930539 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37cb32-208d-480a-8438-37243d2ffdf2 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:53.935004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a66139-b9e5-4db6-931a-b4c03cfb7cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:53.935660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a66139-b9e5-4db6-931a-b4c03cfb7cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:53.935939 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a66139-b9e5-4db6-931a-b4c03cfb7cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:53.936213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a66139-b9e5-4db6-931a-b4c03cfb7cf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:53.936932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a66139-b9e5-4db6-931a-b4c03cfb7cf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:15:53.937230 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a66139-b9e5-4db6-931a-b4c03cfb7cf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:53.939023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a66139-b9e5-4db6-931a-b4c03cfb7cf1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:53.956176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1309/2613] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:15:53.957454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe426c-b432-416a-b95d-ba7c92ad2604 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:53.967943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe426c-b432-416a-b95d-ba7c92ad2604 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:53.973297 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abf969-c0d4-42f6-876c-c4dd506e3a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:53.974000 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abf969-c0d4-42f6-876c-c4dd506e3a7c admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:53.974283 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abf969-c0d4-42f6-876c-c4dd506e3a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:53.974540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abf969-c0d4-42f6-876c-c4dd506e3a7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:53.975253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abf969-c0d4-42f6-876c-c4dd506e3a7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:53.977165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abf969-c0d4-42f6-876c-c4dd506e3a7c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:54.005857 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1305/2614] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:53 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:54.009924 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a66139-b9e5-4db6-931a-b4c03cfb7cf1 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:54.020453 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a66139-b9e5-4db6-931a-b4c03cfb7cf1 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:54.079508 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9780b6-a285-4fd8-83dc-51c7f760542d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:54.080175 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9780b6-a285-4fd8-83dc-51c7f760542d admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:54.080459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9780b6-a285-4fd8-83dc-51c7f760542d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:54.080734 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9780b6-a285-4fd8-83dc-51c7f760542d admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:54.081456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9780b6-a285-4fd8-83dc-51c7f760542d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:54.083451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9780b6-a285-4fd8-83dc-51c7f760542d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:54.106432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1310/2615] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:54 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:54.110778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abf969-c0d4-42f6-876c-c4dd506e3a7c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:54.122114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abf969-c0d4-42f6-876c-c4dd506e3a7c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:54.127853 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab171329-d4ee-4e3e-bd30-4eeaf25c7c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:54.128556 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab171329-d4ee-4e3e-bd30-4eeaf25c7c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:54.128811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab171329-d4ee-4e3e-bd30-4eeaf25c7c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:54.129087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab171329-d4ee-4e3e-bd30-4eeaf25c7c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:54.130129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab171329-d4ee-4e3e-bd30-4eeaf25c7c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:54.131760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab171329-d4ee-4e3e-bd30-4eeaf25c7c91 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:54.145976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1306/2616] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:15:54.150162 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9780b6-a285-4fd8-83dc-51c7f760542d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:54.160926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9780b6-a285-4fd8-83dc-51c7f760542d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:54.166586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ddc71-634d-4dce-a355-63860c70083f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:54.167260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ddc71-634d-4dce-a355-63860c70083f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:54.167543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ddc71-634d-4dce-a355-63860c70083f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:54.167819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ddc71-634d-4dce-a355-63860c70083f admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:54.168547 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ddc71-634d-4dce-a355-63860c70083f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:54.170288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ddc71-634d-4dce-a355-63860c70083f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:54.180671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1311/2617] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:54.184899 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab171329-d4ee-4e3e-bd30-4eeaf25c7c91 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:54.195443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab171329-d4ee-4e3e-bd30-4eeaf25c7c91 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:54.200398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b638291-aaca-4c22-b9a6-b1fc85d2e045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:54.201275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b638291-aaca-4c22-b9a6-b1fc85d2e045 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:54.201555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b638291-aaca-4c22-b9a6-b1fc85d2e045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:54.201857 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b638291-aaca-4c22-b9a6-b1fc85d2e045 admin admin] PATH_INFO: `/v3/projects/13dbe6e09b854fdbb3e416b0cfa8ba72/users/1787030d162c488cbafa5cfa15b2776a/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:54.216782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b638291-aaca-4c22-b9a6-b1fc85d2e045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13dbe6e09b854fdbb3e416b0cfa8ba72, user_id=1787030d162c488cbafa5cfa15b2776a, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:54.219398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b638291-aaca-4c22-b9a6-b1fc85d2e045 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:54.238919 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1307/2618] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:54 2022] PUT /identity/v3/projects/13dbe6e09b854fdbb3e416b0cfa8ba72/users/1787030d162c488cbafa5cfa15b2776a/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:54.253810 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d67fbe-7668-46bf-9de8-3f164654b32a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:54.254128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d67fbe-7668-46bf-9de8-3f164654b32a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:54.254620 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d67fbe-7668-46bf-9de8-3f164654b32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:54.280741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-80d67fbe-7668-46bf-9de8-3f164654b32a None None] MFA Rules not processed for user `1787030d162c488cbafa5cfa15b2776a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:54.281966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-80d67fbe-7668-46bf-9de8-3f164654b32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:54.349422 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b638291-aaca-4c22-b9a6-b1fc85d2e045 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:54.352301 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8b638291-aaca-4c22-b9a6-b1fc85d2e045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:54.355023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1312/2619] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:54 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:54.368367 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d67fbe-7668-46bf-9de8-3f164654b32a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:54.369895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-80d67fbe-7668-46bf-9de8-3f164654b32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:54.420259 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:54.431239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b638291-aaca-4c22-b9a6-b1fc85d2e045 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:54.437522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:54.438477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:54.438762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:54.439253 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] PATH_INFO: `/v3/users/09c51ab1c00a4eed8870b557ae5c40df` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:54.440457 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c51ab1c00a4eed8870b557ae5c40df)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:54.443265 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:54.448803 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:54.462344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d67fbe-7668-46bf-9de8-3f164654b32a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:54.469463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d672f18-8605-4b85-9a0d-79ef00fb74de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef685a17c804e2bbfea1e07cfdf5cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:54.470157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d672f18-8605-4b85-9a0d-79ef00fb74de service glance] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:54.470443 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d672f18-8605-4b85-9a0d-79ef00fb74de service glance] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:54.470759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d672f18-8605-4b85-9a0d-79ef00fb74de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:54.480209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d672f18-8605-4b85-9a0d-79ef00fb74de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:54.482011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d672f18-8605-4b85-9a0d-79ef00fb74de service glance] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:54.494342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1313/2620] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:54 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:54.506271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c51ab1c00a4eed8870b557ae5c40df'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:54.509089 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c51ab1c00a4eed8870b557ae5c40df'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:54.519330 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c51ab1c00a4eed8870b557ae5c40df'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:54.533707 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c51ab1c00a4eed8870b557ae5c40df'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:54.535176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c51ab1c00a4eed8870b557ae5c40df'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:54.553262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1308/2621] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:54 2022] DELETE /identity/v3/users/09c51ab1c00a4eed8870b557ae5c40df => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:54.558039 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba9572-6f17-4883-b474-80addffafb5d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:54.558164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba9572-6f17-4883-b474-80addffafb5d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:54.558385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba9572-6f17-4883-b474-80addffafb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:54.576715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b0ba9572-6f17-4883-b474-80addffafb5d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:54.577946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b0ba9572-6f17-4883-b474-80addffafb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:54.656863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1314/2622] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:54.670707 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:54.672346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:54.738528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa97825-531e-414a-a0bf-0d60871d52dc admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:54.742779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4541c-c4c0-4472-8a2f-0c6e802944a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:54.743507 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4541c-c4c0-4472-8a2f-0c6e802944a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:54.743866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4541c-c4c0-4472-8a2f-0c6e802944a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:54.744438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4541c-c4c0-4472-8a2f-0c6e802944a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:54.756968 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4541c-c4c0-4472-8a2f-0c6e802944a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:54.759201 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4541c-c4c0-4472-8a2f-0c6e802944a3 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:54.770455 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1309/2623] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:55.115990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba9572-6f17-4883-b474-80addffafb5d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:55.117533 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b0ba9572-6f17-4883-b474-80addffafb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:55.138288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba9572-6f17-4883-b474-80addffafb5d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:55.145529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:55.146465 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:55.146749 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:55.147027 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd admin admin] PATH_INFO: `/v3/projects/adf55a22ec6c40b7bb1e293ab02ee863` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:55.149168 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf55a22ec6c40b7bb1e293ab02ee863)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:55.151416 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:55.217431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf55a22ec6c40b7bb1e293ab02ee863', 'request_id': 'req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:55.223522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf55a22ec6c40b7bb1e293ab02ee863', 'request_id': 'req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:55.229745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf55a22ec6c40b7bb1e293ab02ee863', 'request_id': 'req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:55.243386 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd admin admin] The token cache is being invalidate because project adf55a22ec6c40b7bb1e293ab02ee863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:15:55.243676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:55.244288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-46d23dc2-315b-4cbe-b2d3-1f3a3eed14dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:55.246082 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1315/2624] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:55 2022] DELETE /identity/v3/projects/adf55a22ec6c40b7bb1e293ab02ee863 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:55.251605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd919467-e8f7-4064-a11b-0d15ede749f7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:55.251894 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd919467-e8f7-4064-a11b-0d15ede749f7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:55.252179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd919467-e8f7-4064-a11b-0d15ede749f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:55.270204 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-dd919467-e8f7-4064-a11b-0d15ede749f7 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:55.271407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-dd919467-e8f7-4064-a11b-0d15ede749f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:55.334062 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1310/2625] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:55.343240 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03061a6-2a15-4966-b35c-7bd61c22783c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:55.343527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03061a6-2a15-4966-b35c-7bd61c22783c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:55.343804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03061a6-2a15-4966-b35c-7bd61c22783c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:55.364386 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a03061a6-2a15-4966-b35c-7bd61c22783c None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:55.365700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a03061a6-2a15-4966-b35c-7bd61c22783c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:55.373876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1316/2626] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:55.378668 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd919467-e8f7-4064-a11b-0d15ede749f7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:55.390299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd919467-e8f7-4064-a11b-0d15ede749f7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:55.396486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85442601-381f-4e2c-a35b-a3d33072a9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:55.397844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85442601-381f-4e2c-a35b-a3d33072a9f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:55.398132 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85442601-381f-4e2c-a35b-a3d33072a9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:55.398590 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85442601-381f-4e2c-a35b-a3d33072a9f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:55.399339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85442601-381f-4e2c-a35b-a3d33072a9f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:15:55.399651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85442601-381f-4e2c-a35b-a3d33072a9f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:55.401499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85442601-381f-4e2c-a35b-a3d33072a9f5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:55.413971 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1311/2627] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:15:55.418394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03061a6-2a15-4966-b35c-7bd61c22783c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:55.428354 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03061a6-2a15-4966-b35c-7bd61c22783c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:55.433043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ccffc-b72b-4ae3-871e-a85f5abcf3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:55.433845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ccffc-b72b-4ae3-871e-a85f5abcf3be admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:55.434140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ccffc-b72b-4ae3-871e-a85f5abcf3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:55.434415 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ccffc-b72b-4ae3-871e-a85f5abcf3be admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:55.435124 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ccffc-b72b-4ae3-871e-a85f5abcf3be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:55.437048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ccffc-b72b-4ae3-871e-a85f5abcf3be admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:55.464761 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1317/2628] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:55 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:55.469226 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85442601-381f-4e2c-a35b-a3d33072a9f5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:55.481557 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85442601-381f-4e2c-a35b-a3d33072a9f5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:55.491901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:55.544080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf3458-ab9e-49bf-9973-d18ea8ca43de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:55.544793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf3458-ab9e-49bf-9973-d18ea8ca43de admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:55.545085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf3458-ab9e-49bf-9973-d18ea8ca43de admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:55.545336 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf3458-ab9e-49bf-9973-d18ea8ca43de admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:55.546091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf3458-ab9e-49bf-9973-d18ea8ca43de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:55.548031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf3458-ab9e-49bf-9973-d18ea8ca43de admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:55.568300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1312/2629] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:55 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:55.572519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ccffc-b72b-4ae3-871e-a85f5abcf3be admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:55.584362 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ccffc-b72b-4ae3-871e-a85f5abcf3be admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:55.590519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d18960-87e8-445b-9cc8-c8ef58938156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:55.591215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d18960-87e8-445b-9cc8-c8ef58938156 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:55.591500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d18960-87e8-445b-9cc8-c8ef58938156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:55.591776 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d18960-87e8-445b-9cc8-c8ef58938156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:55.592514 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d18960-87e8-445b-9cc8-c8ef58938156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:55.594204 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d18960-87e8-445b-9cc8-c8ef58938156 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:55.606272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1318/2630] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:55.610632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf3458-ab9e-49bf-9973-d18ea8ca43de admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:55.621846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf3458-ab9e-49bf-9973-d18ea8ca43de admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:55.627756 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b38191-9099-45df-8d74-98d7674c710e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:55.628446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b38191-9099-45df-8d74-98d7674c710e admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:55.628794 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b38191-9099-45df-8d74-98d7674c710e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:55.629182 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b38191-9099-45df-8d74-98d7674c710e admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:55.630197 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b38191-9099-45df-8d74-98d7674c710e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:55.631824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b38191-9099-45df-8d74-98d7674c710e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:55.642360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1313/2631] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:55.646667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d18960-87e8-445b-9cc8-c8ef58938156 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:55.658256 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d18960-87e8-445b-9cc8-c8ef58938156 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:55.663920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4807f-a425-4cff-8f72-c9a9206bcf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:55.664819 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4807f-a425-4cff-8f72-c9a9206bcf63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:55.665100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4807f-a425-4cff-8f72-c9a9206bcf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:55.665377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4807f-a425-4cff-8f72-c9a9206bcf63 admin admin] PATH_INFO: `/v3/projects/917c8435f1b74a298c3992bad103aa13/users/a01c7883cb884eabb48f60eee39c5a43/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:55.667693 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:55.682810 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4807f-a425-4cff-8f72-c9a9206bcf63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=917c8435f1b74a298c3992bad103aa13, user_id=a01c7883cb884eabb48f60eee39c5a43, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:55.685507 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4807f-a425-4cff-8f72-c9a9206bcf63 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:55.706723 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1319/2632] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:55 2022] PUT /identity/v3/projects/917c8435f1b74a298c3992bad103aa13/users/a01c7883cb884eabb48f60eee39c5a43/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:55.717794 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d12167-73fa-4f2f-b822-fb67523da16b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:55.718331 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d12167-73fa-4f2f-b822-fb67523da16b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:55.718331 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d12167-73fa-4f2f-b822-fb67523da16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:55.747382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e4d12167-73fa-4f2f-b822-fb67523da16b None None] MFA Rules not processed for user `a01c7883cb884eabb48f60eee39c5a43`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:55.748615 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e4d12167-73fa-4f2f-b822-fb67523da16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:55.826163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1314/2633] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:55.831178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4807f-a425-4cff-8f72-c9a9206bcf63 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:55.901385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4807f-a425-4cff-8f72-c9a9206bcf63 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:55.906440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac1a6a-7389-45e5-9fdf-6b63d2f22fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:55.907113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac1a6a-7389-45e5-9fdf-6b63d2f22fbb admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:55.907444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac1a6a-7389-45e5-9fdf-6b63d2f22fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:55.907753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac1a6a-7389-45e5-9fdf-6b63d2f22fbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:55.908442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac1a6a-7389-45e5-9fdf-6b63d2f22fbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:55.910491 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac1a6a-7389-45e5-9fdf-6b63d2f22fbb admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:55.940681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1320/2634] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:15:55 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:15:55.944603 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d12167-73fa-4f2f-b822-fb67523da16b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:55.960818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d12167-73fa-4f2f-b822-fb67523da16b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:56.033903 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf98105-2e63-47fe-8402-e7a1699107d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:56.034941 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf98105-2e63-47fe-8402-e7a1699107d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:56.035244 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf98105-2e63-47fe-8402-e7a1699107d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:56.035486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf98105-2e63-47fe-8402-e7a1699107d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:56.036415 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf98105-2e63-47fe-8402-e7a1699107d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:56.038573 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf98105-2e63-47fe-8402-e7a1699107d8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:56.071794 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1315/2635] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:15:55 2022] POST /identity/v3/users => generated 294 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:15:56.075384 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac1a6a-7389-45e5-9fdf-6b63d2f22fbb admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:56.087337 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac1a6a-7389-45e5-9fdf-6b63d2f22fbb admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:56.093084 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf3b2f-0fcc-40dc-8b04-0453368a82cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:56.093836 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3b2f-0fcc-40dc-8b04-0453368a82cd admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:56.094192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3b2f-0fcc-40dc-8b04-0453368a82cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:56.094472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3b2f-0fcc-40dc-8b04-0453368a82cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:56.095368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf3b2f-0fcc-40dc-8b04-0453368a82cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:56.097024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf3b2f-0fcc-40dc-8b04-0453368a82cd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:56.109093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1321/2636] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:56.113665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf98105-2e63-47fe-8402-e7a1699107d8 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:56.125607 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf98105-2e63-47fe-8402-e7a1699107d8 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:56.134420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6dd9a-e3f8-4432-85e7-6a50e470d13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:56.135611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6dd9a-e3f8-4432-85e7-6a50e470d13f admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:56.135791 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6dd9a-e3f8-4432-85e7-6a50e470d13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:56.136049 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6dd9a-e3f8-4432-85e7-6a50e470d13f admin admin] PATH_INFO: `/v3/projects/7900dc79dad14c288404d5a4ee6cb688/users/6692beb4f6b240c9bc678e1a34949760/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:56.147655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:56.160668 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6dd9a-e3f8-4432-85e7-6a50e470d13f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7900dc79dad14c288404d5a4ee6cb688, user_id=6692beb4f6b240c9bc678e1a34949760, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:56.163453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6dd9a-e3f8-4432-85e7-6a50e470d13f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:56.177643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1316/2637] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:56 2022] PUT /identity/v3/projects/7900dc79dad14c288404d5a4ee6cb688/users/6692beb4f6b240c9bc678e1a34949760/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:56.181869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf3b2f-0fcc-40dc-8b04-0453368a82cd admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:56.197184 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf3b2f-0fcc-40dc-8b04-0453368a82cd admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:56.266516 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed705185-4341-4416-96ee-dabdb7112854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:56.267194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed705185-4341-4416-96ee-dabdb7112854 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:56.267473 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed705185-4341-4416-96ee-dabdb7112854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:56.267743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed705185-4341-4416-96ee-dabdb7112854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:56.268461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed705185-4341-4416-96ee-dabdb7112854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:56.270445 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed705185-4341-4416-96ee-dabdb7112854 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:56.281101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1322/2638] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:15:56.285500 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6dd9a-e3f8-4432-85e7-6a50e470d13f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:56.298014 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6dd9a-e3f8-4432-85e7-6a50e470d13f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:56.313280 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39435429-b3d9-4442-b625-5f10e6ce1456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:56.314249 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39435429-b3d9-4442-b625-5f10e6ce1456 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:56.314753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39435429-b3d9-4442-b625-5f10e6ce1456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:56.315037 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39435429-b3d9-4442-b625-5f10e6ce1456 admin admin] PATH_INFO: `/v3/projects/7900dc79dad14c288404d5a4ee6cb688/users/6692beb4f6b240c9bc678e1a34949760/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:56.317465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39435429-b3d9-4442-b625-5f10e6ce1456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7900dc79dad14c288404d5a4ee6cb688, user_id=6692beb4f6b240c9bc678e1a34949760, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:56.320167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39435429-b3d9-4442-b625-5f10e6ce1456 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:56.332804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1317/2639] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:15:56 2022] PUT /identity/v3/projects/7900dc79dad14c288404d5a4ee6cb688/users/6692beb4f6b240c9bc678e1a34949760/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:15:56.343952 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567493de-bb2f-4771-98c6-4a18c0f096ca None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:56.344614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567493de-bb2f-4771-98c6-4a18c0f096ca None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:56.344926 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567493de-bb2f-4771-98c6-4a18c0f096ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:56.374924 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-567493de-bb2f-4771-98c6-4a18c0f096ca None None] MFA Rules not processed for user `6692beb4f6b240c9bc678e1a34949760`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:56.376206 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-567493de-bb2f-4771-98c6-4a18c0f096ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:56.419135 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:56.449869 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1323/2640] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:56.461778 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39435429-b3d9-4442-b625-5f10e6ce1456 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:56.464337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-39435429-b3d9-4442-b625-5f10e6ce1456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:56.531723 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39435429-b3d9-4442-b625-5f10e6ce1456 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:56.536034 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab161556-fccb-4078-81d2-c9deca167e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:56.536710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab161556-fccb-4078-81d2-c9deca167e86 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:56.536994 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab161556-fccb-4078-81d2-c9deca167e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:56.537273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab161556-fccb-4078-81d2-c9deca167e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:56.597611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab161556-fccb-4078-81d2-c9deca167e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:56.599172 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab161556-fccb-4078-81d2-c9deca167e86 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:56.608676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1318/2641] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:59.170786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567493de-bb2f-4771-98c6-4a18c0f096ca None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:59.243225 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567493de-bb2f-4771-98c6-4a18c0f096ca None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:59.248138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:59.249121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:59.249541 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:59.249881 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] PATH_INFO: `/v3/users/a01c7883cb884eabb48f60eee39c5a43` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:59.251214 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a01c7883cb884eabb48f60eee39c5a43)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:59.253178 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:59.304501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a01c7883cb884eabb48f60eee39c5a43'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:59.305148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a01c7883cb884eabb48f60eee39c5a43'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:59.314280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a01c7883cb884eabb48f60eee39c5a43'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:59.328744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a01c7883cb884eabb48f60eee39c5a43'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:59.329471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a01c7883cb884eabb48f60eee39c5a43'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:59.343876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1324/2642] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:15:59 2022] DELETE /identity/v3/users/a01c7883cb884eabb48f60eee39c5a43 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:15:59.348737 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aab2d8-e424-42e5-8b0d-03637605bd62 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:59.349228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aab2d8-e424-42e5-8b0d-03637605bd62 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:59.349757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aab2d8-e424-42e5-8b0d-03637605bd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:59.367211 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-56aab2d8-e424-42e5-8b0d-03637605bd62 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:15:59.368425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-56aab2d8-e424-42e5-8b0d-03637605bd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:59.436647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1319/2643] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:15:59.447288 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:59.449551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:59.512491 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6102286-f822-4a34-a4b6-f3e418b9d899 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:59.516344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314ddbc-06d9-4ac8-b732-52be377b62e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:59.517012 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314ddbc-06d9-4ac8-b732-52be377b62e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:59.517295 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314ddbc-06d9-4ac8-b732-52be377b62e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:59.517573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314ddbc-06d9-4ac8-b732-52be377b62e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:59.527482 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314ddbc-06d9-4ac8-b732-52be377b62e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:59.528941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314ddbc-06d9-4ac8-b732-52be377b62e4 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:59.534142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:59.539658 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1325/2644] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:15:59.845990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aab2d8-e424-42e5-8b0d-03637605bd62 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:15:59.848102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-56aab2d8-e424-42e5-8b0d-03637605bd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:15:59.869745 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aab2d8-e424-42e5-8b0d-03637605bd62 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:15:59.882751 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:15:59.884055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:15:59.885171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:15:59.886539 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] PATH_INFO: `/v3/projects/917c8435f1b74a298c3992bad103aa13` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:15:59.888427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] RBAC: Authorizing `identity:delete_project(project_id=917c8435f1b74a298c3992bad103aa13)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:15:59.891846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:15:59.930908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:15:59.971874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '917c8435f1b74a298c3992bad103aa13', 'request_id': 'req-89bdce1a-c953-4f15-af00-cc47b65e574a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:59.979855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '917c8435f1b74a298c3992bad103aa13', 'request_id': 'req-89bdce1a-c953-4f15-af00-cc47b65e574a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:15:59.986504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '917c8435f1b74a298c3992bad103aa13', 'request_id': 'req-89bdce1a-c953-4f15-af00-cc47b65e574a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:00.002498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] The token cache is being invalidate because project 917c8435f1b74a298c3992bad103aa13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:16:00.002927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:00.003637 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:00.005456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1320/2645] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:15:59 2022] DELETE /identity/v3/projects/917c8435f1b74a298c3992bad103aa13 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:00.009788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314ddbc-06d9-4ac8-b732-52be377b62e4 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:00.025377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314ddbc-06d9-4ac8-b732-52be377b62e4 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:00.152237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5d48e-6b24-4644-b35c-0d2169cdbdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:00.153834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5d48e-6b24-4644-b35c-0d2169cdbdc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:00.154177 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5d48e-6b24-4644-b35c-0d2169cdbdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:00.154854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5d48e-6b24-4644-b35c-0d2169cdbdc7 admin admin] PATH_INFO: `/v3/users/6692beb4f6b240c9bc678e1a34949760` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:00.157437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5d48e-6b24-4644-b35c-0d2169cdbdc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6692beb4f6b240c9bc678e1a34949760)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:00.160289 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5d48e-6b24-4644-b35c-0d2169cdbdc7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:00.222886 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-58c5d48e-6b24-4644-b35c-0d2169cdbdc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6692beb4f6b240c9bc678e1a34949760'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:00.223807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-58c5d48e-6b24-4644-b35c-0d2169cdbdc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6692beb4f6b240c9bc678e1a34949760'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:00.236125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-58c5d48e-6b24-4644-b35c-0d2169cdbdc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6692beb4f6b240c9bc678e1a34949760'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:00.250998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-58c5d48e-6b24-4644-b35c-0d2169cdbdc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6692beb4f6b240c9bc678e1a34949760'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:00.251858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-58c5d48e-6b24-4644-b35c-0d2169cdbdc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6692beb4f6b240c9bc678e1a34949760'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:00.273654 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1326/2646] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:16:00 2022] DELETE /identity/v3/users/6692beb4f6b240c9bc678e1a34949760 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:00.605714 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:00.607252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:00.626277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdce1a-c953-4f15-af00-cc47b65e574a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:00.683708 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:00.684577 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:00.684856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:00.685129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a admin admin] PATH_INFO: `/v3/projects/7900dc79dad14c288404d5a4ee6cb688` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:00.686294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7900dc79dad14c288404d5a4ee6cb688)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:00.688186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:00.737927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7900dc79dad14c288404d5a4ee6cb688', 'request_id': 'req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:00.744961 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7900dc79dad14c288404d5a4ee6cb688', 'request_id': 'req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:00.750410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7900dc79dad14c288404d5a4ee6cb688', 'request_id': 'req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:00.765416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a admin admin] The token cache is being invalidate because project 7900dc79dad14c288404d5a4ee6cb688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:16:00.765758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:00.766488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d78fb25d-85be-4cdf-a1ce-0e9b32040b2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:00.768334 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1321/2647] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:16:00 2022] DELETE /identity/v3/projects/7900dc79dad14c288404d5a4ee6cb688 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:16:00.773753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34ce14-0a7f-47f8-9af1-2823405127af None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:00.774181 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34ce14-0a7f-47f8-9af1-2823405127af None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:00.774610 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34ce14-0a7f-47f8-9af1-2823405127af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:00.798030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-9f34ce14-0a7f-47f8-9af1-2823405127af None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:00.799434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9f34ce14-0a7f-47f8-9af1-2823405127af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:00.865198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1327/2648] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:00.874981 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f8347-fe9a-40a9-8997-d22230fbc935 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:00.875325 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f8347-fe9a-40a9-8997-d22230fbc935 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:00.875611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f8347-fe9a-40a9-8997-d22230fbc935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:00.898155 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-6b3f8347-fe9a-40a9-8997-d22230fbc935 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:00.899618 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6b3f8347-fe9a-40a9-8997-d22230fbc935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:00.908930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1322/2649] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:00.913876 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34ce14-0a7f-47f8-9af1-2823405127af None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:00.924572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34ce14-0a7f-47f8-9af1-2823405127af None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:00.930941 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade19b-7848-4bd7-8f67-880bf3e941c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:00.931636 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade19b-7848-4bd7-8f67-880bf3e941c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:00.931910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade19b-7848-4bd7-8f67-880bf3e941c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:00.932183 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade19b-7848-4bd7-8f67-880bf3e941c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:00.932925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ade19b-7848-4bd7-8f67-880bf3e941c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:16:00.933233 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ade19b-7848-4bd7-8f67-880bf3e941c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:00.934910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ade19b-7848-4bd7-8f67-880bf3e941c4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:00.947689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1328/2650] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:16:00.956114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f8347-fe9a-40a9-8997-d22230fbc935 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:00.968652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f8347-fe9a-40a9-8997-d22230fbc935 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:00.973889 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfc601-99b4-4632-945a-ab4620ff3a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:00.974571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfc601-99b4-4632-945a-ab4620ff3a42 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:00.974964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfc601-99b4-4632-945a-ab4620ff3a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:00.975116 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfc601-99b4-4632-945a-ab4620ff3a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:00.975824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdfc601-99b4-4632-945a-ab4620ff3a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:00.977803 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdfc601-99b4-4632-945a-ab4620ff3a42 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:01.009607 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1323/2651] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:16:00 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:16:01.013494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade19b-7848-4bd7-8f67-880bf3e941c4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:01.027118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade19b-7848-4bd7-8f67-880bf3e941c4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:01.031271 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:01.089781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33d5a8-816a-49f6-b0be-e87444f00736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:01.090458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33d5a8-816a-49f6-b0be-e87444f00736 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:01.090804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33d5a8-816a-49f6-b0be-e87444f00736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:01.091189 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33d5a8-816a-49f6-b0be-e87444f00736 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:01.091971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33d5a8-816a-49f6-b0be-e87444f00736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:01.094382 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33d5a8-816a-49f6-b0be-e87444f00736 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:01.118919 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1329/2652] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:16:01 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:16:01.122572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfc601-99b4-4632-945a-ab4620ff3a42 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:01.133447 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfc601-99b4-4632-945a-ab4620ff3a42 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:01.138496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58221d2b-efa3-4c5b-bf4b-6abbb8e9ab12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:01.139167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58221d2b-efa3-4c5b-bf4b-6abbb8e9ab12 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:01.139446 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58221d2b-efa3-4c5b-bf4b-6abbb8e9ab12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:01.139715 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58221d2b-efa3-4c5b-bf4b-6abbb8e9ab12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:01.140472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58221d2b-efa3-4c5b-bf4b-6abbb8e9ab12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:01.142207 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58221d2b-efa3-4c5b-bf4b-6abbb8e9ab12 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:01.155239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1324/2653] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:16:01.159616 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33d5a8-816a-49f6-b0be-e87444f00736 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:01.170213 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33d5a8-816a-49f6-b0be-e87444f00736 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:01.175923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418440c7-a0ed-45cf-b2dd-c8ea256c714c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:01.176610 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418440c7-a0ed-45cf-b2dd-c8ea256c714c admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:01.176891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418440c7-a0ed-45cf-b2dd-c8ea256c714c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:01.177161 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418440c7-a0ed-45cf-b2dd-c8ea256c714c admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:01.177918 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418440c7-a0ed-45cf-b2dd-c8ea256c714c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:01.179687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418440c7-a0ed-45cf-b2dd-c8ea256c714c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:01.189583 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1330/2654] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:16:01.194101 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58221d2b-efa3-4c5b-bf4b-6abbb8e9ab12 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:01.200011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:01.206571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58221d2b-efa3-4c5b-bf4b-6abbb8e9ab12 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:01.213756 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e02645-3b3a-412c-b6fa-f70894363343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:01.214872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e02645-3b3a-412c-b6fa-f70894363343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:01.215387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e02645-3b3a-412c-b6fa-f70894363343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:01.216127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e02645-3b3a-412c-b6fa-f70894363343 admin admin] PATH_INFO: `/v3/projects/132ca9bef5b24e9aa78e7039342aa496/users/1569e2759e3144b3b172ae59e9075d05/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:01.232726 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e02645-3b3a-412c-b6fa-f70894363343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=132ca9bef5b24e9aa78e7039342aa496, user_id=1569e2759e3144b3b172ae59e9075d05, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:01.235504 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e02645-3b3a-412c-b6fa-f70894363343 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:01.251462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1325/2655] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:16:01 2022] PUT /identity/v3/projects/132ca9bef5b24e9aa78e7039342aa496/users/1569e2759e3144b3b172ae59e9075d05/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:01.262136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b565ec9-fd70-42d9-9f11-c3c964417bf2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:01.262667 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b565ec9-fd70-42d9-9f11-c3c964417bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:01.262784 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b565ec9-fd70-42d9-9f11-c3c964417bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:01.289865 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7b565ec9-fd70-42d9-9f11-c3c964417bf2 None None] MFA Rules not processed for user `1569e2759e3144b3b172ae59e9075d05`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:01.291076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7b565ec9-fd70-42d9-9f11-c3c964417bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:01.366920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1331/2656] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:16:01.378158 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e02645-3b3a-412c-b6fa-f70894363343 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:01.379894 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-18e02645-3b3a-412c-b6fa-f70894363343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:01.443952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e02645-3b3a-412c-b6fa-f70894363343 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:01.448288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c175d05-51f9-488d-8c44-e391a9ae264c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:01.449090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c175d05-51f9-488d-8c44-e391a9ae264c service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:01.449442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c175d05-51f9-488d-8c44-e391a9ae264c service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:01.449863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c175d05-51f9-488d-8c44-e391a9ae264c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:01.463820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c175d05-51f9-488d-8c44-e391a9ae264c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:01.465284 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c175d05-51f9-488d-8c44-e391a9ae264c service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:01.475851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1326/2657] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:16:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:16:01.501800 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b565ec9-fd70-42d9-9f11-c3c964417bf2 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:01.591999 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b565ec9-fd70-42d9-9f11-c3c964417bf2 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:01.598406 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:01.599863 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:01.600550 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:01.600902 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] PATH_INFO: `/v3/users/1569e2759e3144b3b172ae59e9075d05` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:01.602643 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1569e2759e3144b3b172ae59e9075d05)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:01.604946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:01.625469 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:01.666460 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1569e2759e3144b3b172ae59e9075d05'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:01.667102 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1569e2759e3144b3b172ae59e9075d05'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:01.674100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1569e2759e3144b3b172ae59e9075d05'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:01.685736 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1569e2759e3144b3b172ae59e9075d05'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:01.686391 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1569e2759e3144b3b172ae59e9075d05'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:01.698747 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1332/2658] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:16:01 2022] DELETE /identity/v3/users/1569e2759e3144b3b172ae59e9075d05 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:01.703579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2386ca-db3d-4ae4-aee6-a8a9e636858e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:01.704087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2386ca-db3d-4ae4-aee6-a8a9e636858e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:01.704087 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2386ca-db3d-4ae4-aee6-a8a9e636858e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:01.722548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2d2386ca-db3d-4ae4-aee6-a8a9e636858e None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:01.723753 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2d2386ca-db3d-4ae4-aee6-a8a9e636858e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:01.786005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1327/2659] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:01.796714 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:01.798284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:01.863248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268d3a6-b89b-49a0-a0b3-4bd2c5c63a3f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:01.867359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f261edf-32fe-4380-b2d6-66210db564c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:01.868074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f261edf-32fe-4380-b2d6-66210db564c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:01.868352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f261edf-32fe-4380-b2d6-66210db564c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:01.869126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f261edf-32fe-4380-b2d6-66210db564c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:01.879210 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f261edf-32fe-4380-b2d6-66210db564c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:01.880706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f261edf-32fe-4380-b2d6-66210db564c1 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:01.891722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1333/2660] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:16:02.185641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2386ca-db3d-4ae4-aee6-a8a9e636858e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:02.187399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2d2386ca-db3d-4ae4-aee6-a8a9e636858e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:02.201806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2386ca-db3d-4ae4-aee6-a8a9e636858e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:02.206415 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:02.207296 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:02.208108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:02.208558 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915 admin admin] PATH_INFO: `/v3/projects/132ca9bef5b24e9aa78e7039342aa496` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:02.210003 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=132ca9bef5b24e9aa78e7039342aa496)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:02.212136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:02.265734 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '132ca9bef5b24e9aa78e7039342aa496', 'request_id': 'req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:02.275620 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '132ca9bef5b24e9aa78e7039342aa496', 'request_id': 'req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:02.281734 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '132ca9bef5b24e9aa78e7039342aa496', 'request_id': 'req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:02.297561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915 admin admin] The token cache is being invalidate because project 132ca9bef5b24e9aa78e7039342aa496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:16:02.297783 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:02.298425 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3ff2f5aa-49b8-4ac3-be6e-5f95ce928915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:02.299999 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1328/2661] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:16:02 2022] DELETE /identity/v3/projects/132ca9bef5b24e9aa78e7039342aa496 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:02.306259 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d10b0-8766-42d1-8639-c54e71c330c9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:02.306834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d10b0-8766-42d1-8639-c54e71c330c9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:02.306989 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d10b0-8766-42d1-8639-c54e71c330c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:02.329352 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-675d10b0-8766-42d1-8639-c54e71c330c9 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:02.330606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-675d10b0-8766-42d1-8639-c54e71c330c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:02.398004 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1334/2662] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:02.405930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50bf94-8f1f-47f0-b43a-a3e3536d0604 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:02.406221 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50bf94-8f1f-47f0-b43a-a3e3536d0604 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:02.406527 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50bf94-8f1f-47f0-b43a-a3e3536d0604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:02.415115 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:02.427703 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-8c50bf94-8f1f-47f0-b43a-a3e3536d0604 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:02.429952 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8c50bf94-8f1f-47f0-b43a-a3e3536d0604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:02.438907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1329/2663] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:02.443488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d10b0-8766-42d1-8639-c54e71c330c9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:02.453506 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d10b0-8766-42d1-8639-c54e71c330c9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:02.458309 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c0cff-a96b-4ed2-8d9f-93f80f4cd83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:02.458974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c0cff-a96b-4ed2-8d9f-93f80f4cd83e admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:02.459266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c0cff-a96b-4ed2-8d9f-93f80f4cd83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:02.459613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c0cff-a96b-4ed2-8d9f-93f80f4cd83e admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:02.460486 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c0cff-a96b-4ed2-8d9f-93f80f4cd83e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:16:02.460793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c0cff-a96b-4ed2-8d9f-93f80f4cd83e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:02.462797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c0cff-a96b-4ed2-8d9f-93f80f4cd83e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:02.474325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1335/2664] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:16:02.478999 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50bf94-8f1f-47f0-b43a-a3e3536d0604 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:02.488915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50bf94-8f1f-47f0-b43a-a3e3536d0604 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:02.494441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d893c0c-c6ac-4fac-8936-8491588c8e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:02.495114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d893c0c-c6ac-4fac-8936-8491588c8e23 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:02.495399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d893c0c-c6ac-4fac-8936-8491588c8e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:02.495677 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d893c0c-c6ac-4fac-8936-8491588c8e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:02.496373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d893c0c-c6ac-4fac-8936-8491588c8e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:02.498347 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d893c0c-c6ac-4fac-8936-8491588c8e23 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:02.526465 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1330/2665] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:16:02 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:16:02.534294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c0cff-a96b-4ed2-8d9f-93f80f4cd83e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:02.546546 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c0cff-a96b-4ed2-8d9f-93f80f4cd83e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:02.569805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:02.610069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9185a15-22f7-4b2a-8a64-55d0a95f9c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:02.610757 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9185a15-22f7-4b2a-8a64-55d0a95f9c84 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:02.611040 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9185a15-22f7-4b2a-8a64-55d0a95f9c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:02.611318 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9185a15-22f7-4b2a-8a64-55d0a95f9c84 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:02.612007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9185a15-22f7-4b2a-8a64-55d0a95f9c84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:02.613976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9185a15-22f7-4b2a-8a64-55d0a95f9c84 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:02.642699 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1336/2666] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:16:02 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:16:02.644183 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d893c0c-c6ac-4fac-8936-8491588c8e23 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:02.656090 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d893c0c-c6ac-4fac-8936-8491588c8e23 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:02.661642 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d526671-1a80-4033-be46-e9ab0ebbeac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:02.662706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d526671-1a80-4033-be46-e9ab0ebbeac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:02.663481 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d526671-1a80-4033-be46-e9ab0ebbeac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:02.663738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d526671-1a80-4033-be46-e9ab0ebbeac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:02.664709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d526671-1a80-4033-be46-e9ab0ebbeac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:02.666396 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d526671-1a80-4033-be46-e9ab0ebbeac7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:02.684233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1331/2667] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:16:02.688631 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9185a15-22f7-4b2a-8a64-55d0a95f9c84 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:02.699058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9185a15-22f7-4b2a-8a64-55d0a95f9c84 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:02.703870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce1a41-c11d-43f2-87b7-57a28f7737ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:02.704613 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce1a41-c11d-43f2-87b7-57a28f7737ab admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:02.704965 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce1a41-c11d-43f2-87b7-57a28f7737ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:02.705313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce1a41-c11d-43f2-87b7-57a28f7737ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:02.706181 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce1a41-c11d-43f2-87b7-57a28f7737ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:02.707812 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce1a41-c11d-43f2-87b7-57a28f7737ab admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:02.719012 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1337/2668] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:16:02.722835 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d526671-1a80-4033-be46-e9ab0ebbeac7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:02.737295 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d526671-1a80-4033-be46-e9ab0ebbeac7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:02.742641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735f9d1-3617-4817-b96a-38b080fd2e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:02.743543 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735f9d1-3617-4817-b96a-38b080fd2e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:02.743826 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735f9d1-3617-4817-b96a-38b080fd2e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:02.744107 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735f9d1-3617-4817-b96a-38b080fd2e57 admin admin] PATH_INFO: `/v3/projects/10baff2f6e0a421d8050f69d97e6c7c1/users/8f80d61cfbe949a7a55d565de30b63ba/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:02.760084 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735f9d1-3617-4817-b96a-38b080fd2e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10baff2f6e0a421d8050f69d97e6c7c1, user_id=8f80d61cfbe949a7a55d565de30b63ba, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:02.763314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735f9d1-3617-4817-b96a-38b080fd2e57 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:02.780168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1332/2669] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:16:02 2022] PUT /identity/v3/projects/10baff2f6e0a421d8050f69d97e6c7c1/users/8f80d61cfbe949a7a55d565de30b63ba/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:02.793218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3e685-bdc3-4165-98da-66c2e32a8b92 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:02.793442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3e685-bdc3-4165-98da-66c2e32a8b92 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:02.793772 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3e685-bdc3-4165-98da-66c2e32a8b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:02.826162 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f0c3e685-bdc3-4165-98da-66c2e32a8b92 None None] MFA Rules not processed for user `8f80d61cfbe949a7a55d565de30b63ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:02.827346 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f0c3e685-bdc3-4165-98da-66c2e32a8b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:02.892456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1338/2670] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:02.904490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735f9d1-3617-4817-b96a-38b080fd2e57 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:02.906523 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9735f9d1-3617-4817-b96a-38b080fd2e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:02.966286 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735f9d1-3617-4817-b96a-38b080fd2e57 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:02.970160 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e035631-75c5-4310-97a3-dcf497fcdc39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:02.970836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e035631-75c5-4310-97a3-dcf497fcdc39 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:02.971118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e035631-75c5-4310-97a3-dcf497fcdc39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:02.971400 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e035631-75c5-4310-97a3-dcf497fcdc39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:02.980938 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e035631-75c5-4310-97a3-dcf497fcdc39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:02.982405 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e035631-75c5-4310-97a3-dcf497fcdc39 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:02.991650 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1333/2671] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:16:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:16:04.361574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b616ca-7f92-4543-aaa2-e44ac94e988c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:04.362228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b616ca-7f92-4543-aaa2-e44ac94e988c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:04.362228 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b616ca-7f92-4543-aaa2-e44ac94e988c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:04.387185 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-16b616ca-7f92-4543-aaa2-e44ac94e988c None None] MFA Rules not processed for user `8f80d61cfbe949a7a55d565de30b63ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:04.389582 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-16b616ca-7f92-4543-aaa2-e44ac94e988c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:04.400732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1339/2672] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:12.987407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:14.377791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:31.386746 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e035631-75c5-4310-97a3-dcf497fcdc39 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:31.388973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9e035631-75c5-4310-97a3-dcf497fcdc39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:31.516039 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e035631-75c5-4310-97a3-dcf497fcdc39 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:31.535984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4c413-50d9-48cb-9f65-27df951fc345 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:31.537318 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4c413-50d9-48cb-9f65-27df951fc345 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:31.537692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4c413-50d9-48cb-9f65-27df951fc345 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:31.537985 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4c413-50d9-48cb-9f65-27df951fc345 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:31.543062 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-71f4c413-50d9-48cb-9f65-27df951fc345 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:31.558988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4c413-50d9-48cb-9f65-27df951fc345 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:31.561860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4c413-50d9-48cb-9f65-27df951fc345 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:31.574287 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1334/2673] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:16:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:16:37.641217 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b616ca-7f92-4543-aaa2-e44ac94e988c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:37.642755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-16b616ca-7f92-4543-aaa2-e44ac94e988c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:37.766566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b616ca-7f92-4543-aaa2-e44ac94e988c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:37.773501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a427c-bc70-4e8d-a37e-832094e13506 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:37.774346 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a427c-bc70-4e8d-a37e-832094e13506 service placement] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:37.774728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a427c-bc70-4e8d-a37e-832094e13506 service placement] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:37.775103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a427c-bc70-4e8d-a37e-832094e13506 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:37.777556 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-612a427c-bc70-4e8d-a37e-832094e13506 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:37.795073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a427c-bc70-4e8d-a37e-832094e13506 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:37.796991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a427c-bc70-4e8d-a37e-832094e13506 service placement] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:37.815121 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1340/2674] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:16:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:16:38.339005 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0a7ec-a001-4a1e-a113-5d274a60dbc0 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:38.339560 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0a7ec-a001-4a1e-a113-5d274a60dbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:38.340078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0a7ec-a001-4a1e-a113-5d274a60dbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:38.363275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d5f0a7ec-a001-4a1e-a113-5d274a60dbc0 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:38.364844 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d5f0a7ec-a001-4a1e-a113-5d274a60dbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:38.441272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1335/2675] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:16:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:38.500409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a427c-bc70-4e8d-a37e-832094e13506 service placement] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:38.502437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-612a427c-bc70-4e8d-a37e-832094e13506 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:38.581461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a427c-bc70-4e8d-a37e-832094e13506 service placement] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:38.588932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056b1fe-664d-4833-8a98-92210d6c287f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:38.589604 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056b1fe-664d-4833-8a98-92210d6c287f service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:38.589920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056b1fe-664d-4833-8a98-92210d6c287f service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:38.590199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056b1fe-664d-4833-8a98-92210d6c287f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:38.598983 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056b1fe-664d-4833-8a98-92210d6c287f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:38.600595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056b1fe-664d-4833-8a98-92210d6c287f service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:38.610054 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1341/2676] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:16:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 10:16:39.214922 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0a7ec-a001-4a1e-a113-5d274a60dbc0 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:39.216685 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d5f0a7ec-a001-4a1e-a113-5d274a60dbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:39.281840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fe84c-fd7e-43cd-a043-e2735ab64d49 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:39.282330 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fe84c-fd7e-43cd-a043-e2735ab64d49 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:39.282428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fe84c-fd7e-43cd-a043-e2735ab64d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:39.300199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0a7ec-a001-4a1e-a113-5d274a60dbc0 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:39.300596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-402fe84c-fd7e-43cd-a043-e2735ab64d49 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:39.302069 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-402fe84c-fd7e-43cd-a043-e2735ab64d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:39.304935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858a4fc-4869-48ad-b7a1-e308e9cf8100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:39.305959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a4fc-4869-48ad-b7a1-e308e9cf8100 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:39.306420 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a4fc-4869-48ad-b7a1-e308e9cf8100 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:39.307214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a4fc-4869-48ad-b7a1-e308e9cf8100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:39.309443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b858a4fc-4869-48ad-b7a1-e308e9cf8100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:39.309739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1342/2677] 10.222.0.79 () {58 vars in 967 bytes} [Mon Jul 4 10:16:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:16:39.325145 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fe84c-fd7e-43cd-a043-e2735ab64d49 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:39.338711 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fe84c-fd7e-43cd-a043-e2735ab64d49 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:39.344149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc7cb2-30d2-49de-8fc8-af9290f82fae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:39.344799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc7cb2-30d2-49de-8fc8-af9290f82fae service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:39.345089 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc7cb2-30d2-49de-8fc8-af9290f82fae service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:39.345361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc7cb2-30d2-49de-8fc8-af9290f82fae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:39.354875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc7cb2-30d2-49de-8fc8-af9290f82fae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:39.356557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc7cb2-30d2-49de-8fc8-af9290f82fae service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:39.359811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:39.366451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1343/2678] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:16:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 10:16:39.380693 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858a4fc-4869-48ad-b7a1-e308e9cf8100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:39.382819 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858a4fc-4869-48ad-b7a1-e308e9cf8100 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:39.394757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1336/2679] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:16:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:16:39.603852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd4482-53ce-4b79-96b6-ff2814004e90 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:39.604242 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd4482-53ce-4b79-96b6-ff2814004e90 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:39.604656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd4482-53ce-4b79-96b6-ff2814004e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:39.623889 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-29fd4482-53ce-4b79-96b6-ff2814004e90 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:39.625259 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-29fd4482-53ce-4b79-96b6-ff2814004e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:39.633060 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1344/2680] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:16:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:39.651928 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858a4fc-4869-48ad-b7a1-e308e9cf8100 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:39.664183 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858a4fc-4869-48ad-b7a1-e308e9cf8100 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:39.669897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1948f3-5094-48db-9c10-0442603a613e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:39.670795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1948f3-5094-48db-9c10-0442603a613e service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:39.671076 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1948f3-5094-48db-9c10-0442603a613e service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:39.671354 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1948f3-5094-48db-9c10-0442603a613e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:39.680601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1948f3-5094-48db-9c10-0442603a613e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:39.682632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1948f3-5094-48db-9c10-0442603a613e service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:39.692390 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1337/2681] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:16:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 10:16:40.125853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809794f-10c3-40fc-ab8e-f35b732aa7b7 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:40.126485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809794f-10c3-40fc-ab8e-f35b732aa7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:40.127074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809794f-10c3-40fc-ab8e-f35b732aa7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:40.149188 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c809794f-10c3-40fc-ab8e-f35b732aa7b7 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:40.150621 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c809794f-10c3-40fc-ab8e-f35b732aa7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:40.158979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1345/2682] 10.222.0.79 () {58 vars in 967 bytes} [Mon Jul 4 10:16:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:16:40.172929 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1948f3-5094-48db-9c10-0442603a613e service swift] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:40.185856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1948f3-5094-48db-9c10-0442603a613e service swift] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:40.191806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa9d8e-ebdb-44c4-a1ec-95f92b870d1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:40.192466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa9d8e-ebdb-44c4-a1ec-95f92b870d1e service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:40.192745 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa9d8e-ebdb-44c4-a1ec-95f92b870d1e service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:40.193019 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa9d8e-ebdb-44c4-a1ec-95f92b870d1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:40.202037 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa9d8e-ebdb-44c4-a1ec-95f92b870d1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:40.203698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa9d8e-ebdb-44c4-a1ec-95f92b870d1e service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:40.212948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1338/2683] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:16:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 10:16:40.793675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809794f-10c3-40fc-ab8e-f35b732aa7b7 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:40.795125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c809794f-10c3-40fc-ab8e-f35b732aa7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:40.809700 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809794f-10c3-40fc-ab8e-f35b732aa7b7 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:40.815542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:40.816522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:40.816866 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:40.817259 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] PATH_INFO: `/v3/users/baf5475348da4210a8fef2e11407b25e` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:40.820803 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baf5475348da4210a8fef2e11407b25e)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:40.823007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:40.898951 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:40.909482 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baf5475348da4210a8fef2e11407b25e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:40.917572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baf5475348da4210a8fef2e11407b25e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:40.930710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baf5475348da4210a8fef2e11407b25e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:40.950386 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baf5475348da4210a8fef2e11407b25e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:40.951428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baf5475348da4210a8fef2e11407b25e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:40.976119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1346/2684] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:16:40 2022] DELETE /identity/v3/users/baf5475348da4210a8fef2e11407b25e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:40.980877 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac860e4-6751-44d1-8ddb-00a2cd04b56d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:40.981869 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac860e4-6751-44d1-8ddb-00a2cd04b56d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:40.982325 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac860e4-6751-44d1-8ddb-00a2cd04b56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:41.018833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9ac860e4-6751-44d1-8ddb-00a2cd04b56d None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:41.020827 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9ac860e4-6751-44d1-8ddb-00a2cd04b56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:41.118806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1339/2685] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:41.137263 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:41.141681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:41.224994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7406c7-a3e0-43e4-819b-904a8a68b3e8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:41.229644 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6aa10-2d7c-418e-b15b-84dc61ba7bef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:41.230441 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6aa10-2d7c-418e-b15b-84dc61ba7bef service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:41.230746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6aa10-2d7c-418e-b15b-84dc61ba7bef service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:41.231022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6aa10-2d7c-418e-b15b-84dc61ba7bef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:41.242233 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6aa10-2d7c-418e-b15b-84dc61ba7bef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:41.247410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6aa10-2d7c-418e-b15b-84dc61ba7bef service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:41.260294 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1347/2686] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:16:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:16:41.742900 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac860e4-6751-44d1-8ddb-00a2cd04b56d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:41.744438 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9ac860e4-6751-44d1-8ddb-00a2cd04b56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:41.767360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac860e4-6751-44d1-8ddb-00a2cd04b56d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:41.781414 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:41.782383 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:41.782858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:41.783155 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] PATH_INFO: `/v3/projects/17ccee846512409fb020914db8d8a13b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:41.784362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ccee846512409fb020914db8d8a13b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:41.786859 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:41.842158 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ccee846512409fb020914db8d8a13b', 'request_id': 'req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:41.848723 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ccee846512409fb020914db8d8a13b', 'request_id': 'req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:41.855540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ccee846512409fb020914db8d8a13b', 'request_id': 'req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:41.870498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] The token cache is being invalidate because project 17ccee846512409fb020914db8d8a13b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:16:41.870822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:41.871449 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:41.873267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1340/2687] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:16:41 2022] DELETE /identity/v3/projects/17ccee846512409fb020914db8d8a13b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:41.877774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6aa10-2d7c-418e-b15b-84dc61ba7bef service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:41.892444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6aa10-2d7c-418e-b15b-84dc61ba7bef service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:41.994437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc7f36-a482-4dda-bde2-2de6f3f581b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:41.995338 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc7f36-a482-4dda-bde2-2de6f3f581b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:41.995795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc7f36-a482-4dda-bde2-2de6f3f581b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:41.996076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc7f36-a482-4dda-bde2-2de6f3f581b4 admin admin] PATH_INFO: `/v3/users/9a38442b69bf433fa8d3d3a7b14d49e6` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:41.997309 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc7f36-a482-4dda-bde2-2de6f3f581b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a38442b69bf433fa8d3d3a7b14d49e6)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:41.999287 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc7f36-a482-4dda-bde2-2de6f3f581b4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:42.053442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c3cc7f36-a482-4dda-bde2-2de6f3f581b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a38442b69bf433fa8d3d3a7b14d49e6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:42.054292 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c3cc7f36-a482-4dda-bde2-2de6f3f581b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a38442b69bf433fa8d3d3a7b14d49e6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:42.063435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c3cc7f36-a482-4dda-bde2-2de6f3f581b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a38442b69bf433fa8d3d3a7b14d49e6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:42.071420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:42.079049 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c3cc7f36-a482-4dda-bde2-2de6f3f581b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a38442b69bf433fa8d3d3a7b14d49e6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:42.079908 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c3cc7f36-a482-4dda-bde2-2de6f3f581b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a38442b69bf433fa8d3d3a7b14d49e6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:42.093130 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1348/2688] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:16:41 2022] DELETE /identity/v3/users/9a38442b69bf433fa8d3d3a7b14d49e6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:42.427687 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:42.429862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:42.446247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce42525-b3ba-4ced-ae33-814ae3fb9d7b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:42.499880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:42.535906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc47f5-093d-49cc-aec7-d1b14725645f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:42.537067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc47f5-093d-49cc-aec7-d1b14725645f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:42.537348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc47f5-093d-49cc-aec7-d1b14725645f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:42.537645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc47f5-093d-49cc-aec7-d1b14725645f admin admin] PATH_INFO: `/v3/projects/444c9611ae9f4260b580c5814d9d8d3e` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:42.538966 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc47f5-093d-49cc-aec7-d1b14725645f admin admin] RBAC: Authorizing `identity:delete_project(project_id=444c9611ae9f4260b580c5814d9d8d3e)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:42.540980 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc47f5-093d-49cc-aec7-d1b14725645f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:42.596494 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9dbc47f5-093d-49cc-aec7-d1b14725645f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444c9611ae9f4260b580c5814d9d8d3e', 'request_id': 'req-9dbc47f5-093d-49cc-aec7-d1b14725645f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:42.604904 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9dbc47f5-093d-49cc-aec7-d1b14725645f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444c9611ae9f4260b580c5814d9d8d3e', 'request_id': 'req-9dbc47f5-093d-49cc-aec7-d1b14725645f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:42.613490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9dbc47f5-093d-49cc-aec7-d1b14725645f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444c9611ae9f4260b580c5814d9d8d3e', 'request_id': 'req-9dbc47f5-093d-49cc-aec7-d1b14725645f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:42.633017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9dbc47f5-093d-49cc-aec7-d1b14725645f admin admin] The token cache is being invalidate because project 444c9611ae9f4260b580c5814d9d8d3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:16:42.633222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9dbc47f5-093d-49cc-aec7-d1b14725645f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:42.634069 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9dbc47f5-093d-49cc-aec7-d1b14725645f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:42.637636 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1341/2689] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:16:42 2022] DELETE /identity/v3/projects/444c9611ae9f4260b580c5814d9d8d3e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:16:42.643558 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcaa468-7c02-403a-ae4b-acbd33cd2d14 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:42.644080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcaa468-7c02-403a-ae4b-acbd33cd2d14 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:42.644606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcaa468-7c02-403a-ae4b-acbd33cd2d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:42.667777 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4fcaa468-7c02-403a-ae4b-acbd33cd2d14 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:42.670025 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4fcaa468-7c02-403a-ae4b-acbd33cd2d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:42.763802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1349/2690] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:42.772004 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d83d2-bdfb-4aee-be41-e0213824be19 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:42.772299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d83d2-bdfb-4aee-be41-e0213824be19 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:42.772581 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d83d2-bdfb-4aee-be41-e0213824be19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:42.793658 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ac2d83d2-bdfb-4aee-be41-e0213824be19 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:42.795227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ac2d83d2-bdfb-4aee-be41-e0213824be19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:42.807624 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1342/2691] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:42.809342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcaa468-7c02-403a-ae4b-acbd33cd2d14 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:42.819757 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcaa468-7c02-403a-ae4b-acbd33cd2d14 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:42.825017 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c4324-dd22-489f-a093-c4837372c243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:42.826055 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c4324-dd22-489f-a093-c4837372c243 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:42.826559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c4324-dd22-489f-a093-c4837372c243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:42.827041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c4324-dd22-489f-a093-c4837372c243 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:42.828538 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c4324-dd22-489f-a093-c4837372c243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:16:42.829605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c4324-dd22-489f-a093-c4837372c243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:42.831741 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c4324-dd22-489f-a093-c4837372c243 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:42.849976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1350/2692] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:16:42.850912 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d83d2-bdfb-4aee-be41-e0213824be19 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:42.862764 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d83d2-bdfb-4aee-be41-e0213824be19 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:42.873330 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026cc78-347a-4311-8f5a-c5f61d2bf7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:42.874064 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026cc78-347a-4311-8f5a-c5f61d2bf7c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:42.874345 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026cc78-347a-4311-8f5a-c5f61d2bf7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:42.874620 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026cc78-347a-4311-8f5a-c5f61d2bf7c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:42.875329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026cc78-347a-4311-8f5a-c5f61d2bf7c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:42.877247 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026cc78-347a-4311-8f5a-c5f61d2bf7c3 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:42.909381 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1343/2693] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:16:42 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:16:42.913776 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c4324-dd22-489f-a093-c4837372c243 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:42.927056 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c4324-dd22-489f-a093-c4837372c243 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:43.008974 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4aa89-cb23-4866-ad19-c9c67a7314d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:43.009797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4aa89-cb23-4866-ad19-c9c67a7314d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:43.010199 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4aa89-cb23-4866-ad19-c9c67a7314d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:43.010670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4aa89-cb23-4866-ad19-c9c67a7314d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:43.011394 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4aa89-cb23-4866-ad19-c9c67a7314d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:43.013379 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4aa89-cb23-4866-ad19-c9c67a7314d6 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:43.035002 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1351/2694] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:16:42 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:16:43.039481 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026cc78-347a-4311-8f5a-c5f61d2bf7c3 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:43.050392 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026cc78-347a-4311-8f5a-c5f61d2bf7c3 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:43.058522 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c7e2b-5759-4270-9c36-84f8a3c47c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:43.059315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c7e2b-5759-4270-9c36-84f8a3c47c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:43.059666 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c7e2b-5759-4270-9c36-84f8a3c47c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:43.060007 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c7e2b-5759-4270-9c36-84f8a3c47c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:43.060837 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c7e2b-5759-4270-9c36-84f8a3c47c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:43.062633 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c7e2b-5759-4270-9c36-84f8a3c47c85 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:43.074125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1344/2695] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:16:43.078334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4aa89-cb23-4866-ad19-c9c67a7314d6 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:43.090446 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4aa89-cb23-4866-ad19-c9c67a7314d6 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:43.095335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ae3b0-9145-4e9c-83ad-ece14e7ae19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:43.096278 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ae3b0-9145-4e9c-83ad-ece14e7ae19b admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:43.096891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ae3b0-9145-4e9c-83ad-ece14e7ae19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:43.097399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ae3b0-9145-4e9c-83ad-ece14e7ae19b admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:43.098341 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ae3b0-9145-4e9c-83ad-ece14e7ae19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:43.100672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ae3b0-9145-4e9c-83ad-ece14e7ae19b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:43.113202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1352/2696] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:16:43.116301 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c7e2b-5759-4270-9c36-84f8a3c47c85 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:43.138164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c7e2b-5759-4270-9c36-84f8a3c47c85 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:43.147144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916cc57-dbad-456e-9234-bb7d98fa8c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:43.148203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916cc57-dbad-456e-9234-bb7d98fa8c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:43.148488 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916cc57-dbad-456e-9234-bb7d98fa8c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:43.148964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916cc57-dbad-456e-9234-bb7d98fa8c3e admin admin] PATH_INFO: `/v3/projects/45dd96c90c824c45afd3b6b345b47e7d/users/da29008234dc444a8fd99cd5a5bcc4cc/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:43.166544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916cc57-dbad-456e-9234-bb7d98fa8c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=45dd96c90c824c45afd3b6b345b47e7d, user_id=da29008234dc444a8fd99cd5a5bcc4cc, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:43.168685 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916cc57-dbad-456e-9234-bb7d98fa8c3e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:43.192935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1345/2697] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:16:43 2022] PUT /identity/v3/projects/45dd96c90c824c45afd3b6b345b47e7d/users/da29008234dc444a8fd99cd5a5bcc4cc/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:43.203794 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1441f4-53dd-4321-b584-452fbf5b484a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:43.204201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1441f4-53dd-4321-b584-452fbf5b484a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:43.204546 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1441f4-53dd-4321-b584-452fbf5b484a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:43.235024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-0f1441f4-53dd-4321-b584-452fbf5b484a None None] MFA Rules not processed for user `da29008234dc444a8fd99cd5a5bcc4cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:43.236160 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0f1441f4-53dd-4321-b584-452fbf5b484a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:43.269976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:43.303480 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1353/2698] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:43.308368 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916cc57-dbad-456e-9234-bb7d98fa8c3e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:43.370872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916cc57-dbad-456e-9234-bb7d98fa8c3e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:43.375920 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e46dd-107f-4a95-9f54-0fad64de6d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:43.376588 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e46dd-107f-4a95-9f54-0fad64de6d5f admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:43.376871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e46dd-107f-4a95-9f54-0fad64de6d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:43.377147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e46dd-107f-4a95-9f54-0fad64de6d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:43.377882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e46dd-107f-4a95-9f54-0fad64de6d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:43.379822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e46dd-107f-4a95-9f54-0fad64de6d5f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:43.388418 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:43.417442 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1346/2699] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:16:43 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:16:43.421696 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1441f4-53dd-4321-b584-452fbf5b484a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:43.433932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1441f4-53dd-4321-b584-452fbf5b484a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:43.493068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac386e-e6e2-46ce-bc8d-e99d553ef4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:43.493775 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac386e-e6e2-46ce-bc8d-e99d553ef4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:43.494164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac386e-e6e2-46ce-bc8d-e99d553ef4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:43.494879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac386e-e6e2-46ce-bc8d-e99d553ef4e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:43.495247 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac386e-e6e2-46ce-bc8d-e99d553ef4e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:43.497185 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac386e-e6e2-46ce-bc8d-e99d553ef4e8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:43.518117 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1354/2700] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:16:43 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:16:43.524147 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e46dd-107f-4a95-9f54-0fad64de6d5f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:43.535166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e46dd-107f-4a95-9f54-0fad64de6d5f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:43.539935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ce28f-f223-4080-9402-fc6c22935887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:43.540599 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ce28f-f223-4080-9402-fc6c22935887 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:43.540896 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ce28f-f223-4080-9402-fc6c22935887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:43.541202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ce28f-f223-4080-9402-fc6c22935887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:43.541959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ce28f-f223-4080-9402-fc6c22935887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:43.543581 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ce28f-f223-4080-9402-fc6c22935887 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:43.554291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1347/2701] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:16:43.558572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac386e-e6e2-46ce-bc8d-e99d553ef4e8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:43.568603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac386e-e6e2-46ce-bc8d-e99d553ef4e8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:43.573992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8c2a7-0f9f-4eb3-a4f1-d01b088bb05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:43.574878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8c2a7-0f9f-4eb3-a4f1-d01b088bb05a admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:43.575157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8c2a7-0f9f-4eb3-a4f1-d01b088bb05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:43.575431 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8c2a7-0f9f-4eb3-a4f1-d01b088bb05a admin admin] PATH_INFO: `/v3/projects/796a3326c4d74dc68a84cd6fceae7e2f/users/f3019ab9f06143b1aa0391fd80babd06/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:43.590945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8c2a7-0f9f-4eb3-a4f1-d01b088bb05a admin admin] RBAC: Authorizing `identity:create_grant(project_id=796a3326c4d74dc68a84cd6fceae7e2f, user_id=f3019ab9f06143b1aa0391fd80babd06, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:43.593731 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8c2a7-0f9f-4eb3-a4f1-d01b088bb05a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:43.608785 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1355/2702] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:16:43 2022] PUT /identity/v3/projects/796a3326c4d74dc68a84cd6fceae7e2f/users/f3019ab9f06143b1aa0391fd80babd06/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:43.612914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ce28f-f223-4080-9402-fc6c22935887 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:43.623373 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ce28f-f223-4080-9402-fc6c22935887 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:43.679111 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964b01b-f297-4e72-a390-8b644eea9d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:43.679663 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964b01b-f297-4e72-a390-8b644eea9d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:43.679973 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964b01b-f297-4e72-a390-8b644eea9d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:43.680246 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964b01b-f297-4e72-a390-8b644eea9d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:43.680975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964b01b-f297-4e72-a390-8b644eea9d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:43.682788 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964b01b-f297-4e72-a390-8b644eea9d76 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:43.693341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1348/2703] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:16:43.697795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8c2a7-0f9f-4eb3-a4f1-d01b088bb05a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:43.708674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8c2a7-0f9f-4eb3-a4f1-d01b088bb05a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:43.714332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4da73-671c-48ff-9d94-2b45386179be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:43.715259 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4da73-671c-48ff-9d94-2b45386179be admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:43.715545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4da73-671c-48ff-9d94-2b45386179be admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:43.715824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4da73-671c-48ff-9d94-2b45386179be admin admin] PATH_INFO: `/v3/projects/796a3326c4d74dc68a84cd6fceae7e2f/users/f3019ab9f06143b1aa0391fd80babd06/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:43.718370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4da73-671c-48ff-9d94-2b45386179be admin admin] RBAC: Authorizing `identity:create_grant(project_id=796a3326c4d74dc68a84cd6fceae7e2f, user_id=f3019ab9f06143b1aa0391fd80babd06, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:43.721679 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4da73-671c-48ff-9d94-2b45386179be admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:43.734181 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1356/2704] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:16:43 2022] PUT /identity/v3/projects/796a3326c4d74dc68a84cd6fceae7e2f/users/f3019ab9f06143b1aa0391fd80babd06/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:43.746121 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405dda4-bc42-466b-9351-49cef4277b67 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:43.746423 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405dda4-bc42-466b-9351-49cef4277b67 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:43.746710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405dda4-bc42-466b-9351-49cef4277b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:43.773639 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4405dda4-bc42-466b-9351-49cef4277b67 None None] MFA Rules not processed for user `f3019ab9f06143b1aa0391fd80babd06`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:43.774871 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4405dda4-bc42-466b-9351-49cef4277b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:43.847506 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1349/2705] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:43.859019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4da73-671c-48ff-9d94-2b45386179be admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:43.860738 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-45e4da73-671c-48ff-9d94-2b45386179be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:43.923138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4da73-671c-48ff-9d94-2b45386179be admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:43.928152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d834498-6249-4d1e-9084-68481d046579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:43.929668 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d834498-6249-4d1e-9084-68481d046579 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:43.929911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d834498-6249-4d1e-9084-68481d046579 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:43.930193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d834498-6249-4d1e-9084-68481d046579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:43.984440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:44.008931 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d834498-6249-4d1e-9084-68481d046579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:44.011123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d834498-6249-4d1e-9084-68481d046579 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:44.023300 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1357/2706] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:16:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:16:44.484102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405dda4-bc42-466b-9351-49cef4277b67 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:44.485674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4405dda4-bc42-466b-9351-49cef4277b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:44.641773 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405dda4-bc42-466b-9351-49cef4277b67 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:44.646992 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e2e1f-a2c5-4612-9675-add89389b64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:44.647899 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e2e1f-a2c5-4612-9675-add89389b64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:44.648186 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e2e1f-a2c5-4612-9675-add89389b64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:44.648460 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e2e1f-a2c5-4612-9675-add89389b64e admin admin] PATH_INFO: `/v3/users/1787030d162c488cbafa5cfa15b2776a` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:44.650516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e2e1f-a2c5-4612-9675-add89389b64e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1787030d162c488cbafa5cfa15b2776a)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:44.653013 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e2e1f-a2c5-4612-9675-add89389b64e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:44.655233 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:44.767406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-534e2e1f-a2c5-4612-9675-add89389b64e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1787030d162c488cbafa5cfa15b2776a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:44.768500 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-534e2e1f-a2c5-4612-9675-add89389b64e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1787030d162c488cbafa5cfa15b2776a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:44.786123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d834498-6249-4d1e-9084-68481d046579 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:44.787826 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7d834498-6249-4d1e-9084-68481d046579 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:44.789004 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-534e2e1f-a2c5-4612-9675-add89389b64e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1787030d162c488cbafa5cfa15b2776a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:44.816623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-534e2e1f-a2c5-4612-9675-add89389b64e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1787030d162c488cbafa5cfa15b2776a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:44.817839 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-534e2e1f-a2c5-4612-9675-add89389b64e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1787030d162c488cbafa5cfa15b2776a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:44.821797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d834498-6249-4d1e-9084-68481d046579 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:44.834824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbd978-9225-46f0-9aa5-fcc83271b675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:44.839062 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1350/2707] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:16:44 2022] DELETE /identity/v3/users/1787030d162c488cbafa5cfa15b2776a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:44.842118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2543e-d88d-4f3f-8fd4-b18514f84a81 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:44.842410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2543e-d88d-4f3f-8fd4-b18514f84a81 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:44.842692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2543e-d88d-4f3f-8fd4-b18514f84a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:44.852057 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbd978-9225-46f0-9aa5-fcc83271b675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:44.852400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbd978-9225-46f0-9aa5-fcc83271b675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:44.852678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbd978-9225-46f0-9aa5-fcc83271b675 admin admin] PATH_INFO: `/v3/users/8f80d61cfbe949a7a55d565de30b63ba` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:44.854323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbd978-9225-46f0-9aa5-fcc83271b675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f80d61cfbe949a7a55d565de30b63ba)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:44.856321 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbd978-9225-46f0-9aa5-fcc83271b675 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:44.882782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-98a2543e-d88d-4f3f-8fd4-b18514f84a81 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:44.884643 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-98a2543e-d88d-4f3f-8fd4-b18514f84a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:44.970135 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3fcbd978-9225-46f0-9aa5-fcc83271b675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f80d61cfbe949a7a55d565de30b63ba'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:44.971458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3fcbd978-9225-46f0-9aa5-fcc83271b675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f80d61cfbe949a7a55d565de30b63ba'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:44.981875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3fcbd978-9225-46f0-9aa5-fcc83271b675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f80d61cfbe949a7a55d565de30b63ba'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:45.004336 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1351/2708] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:16:45.004575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3fcbd978-9225-46f0-9aa5-fcc83271b675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f80d61cfbe949a7a55d565de30b63ba'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:45.004575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3fcbd978-9225-46f0-9aa5-fcc83271b675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f80d61cfbe949a7a55d565de30b63ba'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:45.013907 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2543e-d88d-4f3f-8fd4-b18514f84a81 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:45.015534 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-98a2543e-d88d-4f3f-8fd4-b18514f84a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:45.031138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1358/2709] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:16:44 2022] DELETE /identity/v3/users/8f80d61cfbe949a7a55d565de30b63ba => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:45.041115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046a50e-c226-4e66-b8a2-8637d45b98bd None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:45.041115 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046a50e-c226-4e66-b8a2-8637d45b98bd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:45.041458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046a50e-c226-4e66-b8a2-8637d45b98bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:45.082858 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2046a50e-c226-4e66-b8a2-8637d45b98bd None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:45.084129 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2046a50e-c226-4e66-b8a2-8637d45b98bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:45.116858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2543e-d88d-4f3f-8fd4-b18514f84a81 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:45.155732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1359/2710] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:45.165728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046a50e-c226-4e66-b8a2-8637d45b98bd None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:45.168443 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2046a50e-c226-4e66-b8a2-8637d45b98bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:45.177308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:45.214514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f944d9d-9e51-4168-9dc0-943bbbc29ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:45.215807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f944d9d-9e51-4168-9dc0-943bbbc29ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:45.216103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f944d9d-9e51-4168-9dc0-943bbbc29ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:45.216389 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f944d9d-9e51-4168-9dc0-943bbbc29ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:45.219329 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9f944d9d-9e51-4168-9dc0-943bbbc29ef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:45.247303 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f944d9d-9e51-4168-9dc0-943bbbc29ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:45.249364 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f944d9d-9e51-4168-9dc0-943bbbc29ef5 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:45.265437 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1352/2711] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:16:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:16:45.284140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046a50e-c226-4e66-b8a2-8637d45b98bd None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:45.290289 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37cd774-3d70-46d7-86f2-7481cdf83c35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:45.291563 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cd774-3d70-46d7-86f2-7481cdf83c35 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:45.291947 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cd774-3d70-46d7-86f2-7481cdf83c35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:45.292439 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cd774-3d70-46d7-86f2-7481cdf83c35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:45.305285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37cd774-3d70-46d7-86f2-7481cdf83c35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:45.307680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37cd774-3d70-46d7-86f2-7481cdf83c35 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:45.313998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:45.324839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1360/2712] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:16:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:16:45.668474 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f944d9d-9e51-4168-9dc0-943bbbc29ef5 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:45.670882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9f944d9d-9e51-4168-9dc0-943bbbc29ef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:45.700882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f944d9d-9e51-4168-9dc0-943bbbc29ef5 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:45.723915 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff93694-fe48-4508-ad3f-40d1a75c10bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:45.725132 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff93694-fe48-4508-ad3f-40d1a75c10bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:45.725597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff93694-fe48-4508-ad3f-40d1a75c10bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:45.725976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff93694-fe48-4508-ad3f-40d1a75c10bb admin admin] PATH_INFO: `/v3/projects/13dbe6e09b854fdbb3e416b0cfa8ba72` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:45.727351 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff93694-fe48-4508-ad3f-40d1a75c10bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=13dbe6e09b854fdbb3e416b0cfa8ba72)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:45.730038 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff93694-fe48-4508-ad3f-40d1a75c10bb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:45.767496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37cd774-3d70-46d7-86f2-7481cdf83c35 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:45.769053 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f37cd774-3d70-46d7-86f2-7481cdf83c35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:45.794261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37cd774-3d70-46d7-86f2-7481cdf83c35 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:45.815862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2ff93694-fe48-4508-ad3f-40d1a75c10bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13dbe6e09b854fdbb3e416b0cfa8ba72', 'request_id': 'req-2ff93694-fe48-4508-ad3f-40d1a75c10bb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:45.823358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2ff93694-fe48-4508-ad3f-40d1a75c10bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13dbe6e09b854fdbb3e416b0cfa8ba72', 'request_id': 'req-2ff93694-fe48-4508-ad3f-40d1a75c10bb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:45.844706 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2ff93694-fe48-4508-ad3f-40d1a75c10bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13dbe6e09b854fdbb3e416b0cfa8ba72', 'request_id': 'req-2ff93694-fe48-4508-ad3f-40d1a75c10bb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:45.868690 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2ff93694-fe48-4508-ad3f-40d1a75c10bb admin admin] The token cache is being invalidate because project 13dbe6e09b854fdbb3e416b0cfa8ba72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:16:45.869041 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2ff93694-fe48-4508-ad3f-40d1a75c10bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:45.871914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2ff93694-fe48-4508-ad3f-40d1a75c10bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:45.876416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1353/2713] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:16:45 2022] DELETE /identity/v3/projects/13dbe6e09b854fdbb3e416b0cfa8ba72 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:45.898165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3dcfd-8a44-42eb-a558-d8703a416766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:45.899184 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3dcfd-8a44-42eb-a558-d8703a416766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:45.899462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3dcfd-8a44-42eb-a558-d8703a416766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:45.899748 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3dcfd-8a44-42eb-a558-d8703a416766 admin admin] PATH_INFO: `/v3/projects/10baff2f6e0a421d8050f69d97e6c7c1` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:45.902105 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3dcfd-8a44-42eb-a558-d8703a416766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10baff2f6e0a421d8050f69d97e6c7c1)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:45.904347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3dcfd-8a44-42eb-a558-d8703a416766 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:45.990776 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5ec3dcfd-8a44-42eb-a558-d8703a416766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10baff2f6e0a421d8050f69d97e6c7c1', 'request_id': 'req-5ec3dcfd-8a44-42eb-a558-d8703a416766'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:46.000378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5ec3dcfd-8a44-42eb-a558-d8703a416766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10baff2f6e0a421d8050f69d97e6c7c1', 'request_id': 'req-5ec3dcfd-8a44-42eb-a558-d8703a416766'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:46.008934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5ec3dcfd-8a44-42eb-a558-d8703a416766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10baff2f6e0a421d8050f69d97e6c7c1', 'request_id': 'req-5ec3dcfd-8a44-42eb-a558-d8703a416766'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:46.042113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5ec3dcfd-8a44-42eb-a558-d8703a416766 admin admin] The token cache is being invalidate because project 10baff2f6e0a421d8050f69d97e6c7c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:16:46.042113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5ec3dcfd-8a44-42eb-a558-d8703a416766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:46.042113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5ec3dcfd-8a44-42eb-a558-d8703a416766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:46.042113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1361/2714] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:16:45 2022] DELETE /identity/v3/projects/10baff2f6e0a421d8050f69d97e6c7c1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:16:46.043692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1322b-428b-483c-8622-64a0a23c1ff7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:46.044074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1322b-428b-483c-8622-64a0a23c1ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:46.044693 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1322b-428b-483c-8622-64a0a23c1ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:46.075960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1de1322b-428b-483c-8622-64a0a23c1ff7 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:46.078085 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1de1322b-428b-483c-8622-64a0a23c1ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:46.182569 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1354/2715] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:16:46.202343 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d155b-62b7-4104-bd02-160446fe4a65 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:46.202759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d155b-62b7-4104-bd02-160446fe4a65 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:46.203131 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d155b-62b7-4104-bd02-160446fe4a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:46.227799 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f85d155b-62b7-4104-bd02-160446fe4a65 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:46.229157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f85d155b-62b7-4104-bd02-160446fe4a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:46.238759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1362/2716] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:46.243105 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1322b-428b-483c-8622-64a0a23c1ff7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:46.253437 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1322b-428b-483c-8622-64a0a23c1ff7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:46.258540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcb5a7-d3c2-4963-b606-7ed1eeb5e90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:46.259322 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcb5a7-d3c2-4963-b606-7ed1eeb5e90a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:46.260318 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcb5a7-d3c2-4963-b606-7ed1eeb5e90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:46.260318 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcb5a7-d3c2-4963-b606-7ed1eeb5e90a admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:46.261324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcb5a7-d3c2-4963-b606-7ed1eeb5e90a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:16:46.261933 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcb5a7-d3c2-4963-b606-7ed1eeb5e90a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:46.263765 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcb5a7-d3c2-4963-b606-7ed1eeb5e90a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:46.284652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1355/2717] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:16:46.289327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d155b-62b7-4104-bd02-160446fe4a65 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:46.300500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d155b-62b7-4104-bd02-160446fe4a65 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:46.306383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe3cef-548c-40c9-9f8d-bf073550d302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:46.306856 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe3cef-548c-40c9-9f8d-bf073550d302 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:46.307260 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe3cef-548c-40c9-9f8d-bf073550d302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:46.307596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe3cef-548c-40c9-9f8d-bf073550d302 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:46.308359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe3cef-548c-40c9-9f8d-bf073550d302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:46.310438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe3cef-548c-40c9-9f8d-bf073550d302 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:46.344450 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1363/2718] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:16:46 2022] POST /identity/v3/projects => generated 395 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:16:46.348828 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcb5a7-d3c2-4963-b606-7ed1eeb5e90a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:46.359451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcb5a7-d3c2-4963-b606-7ed1eeb5e90a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:46.437084 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:46.463872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ae56-1ab8-4393-acd5-340192c9231c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:46.464657 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ae56-1ab8-4393-acd5-340192c9231c admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:46.464843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ae56-1ab8-4393-acd5-340192c9231c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:46.465117 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ae56-1ab8-4393-acd5-340192c9231c admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:46.465904 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609ae56-1ab8-4393-acd5-340192c9231c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:46.467836 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609ae56-1ab8-4393-acd5-340192c9231c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:46.503586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1356/2719] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:16:46 2022] POST /identity/v3/users => generated 298 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:16:46.504299 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe3cef-548c-40c9-9f8d-bf073550d302 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:46.521126 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe3cef-548c-40c9-9f8d-bf073550d302 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:46.532723 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ab7dc-5b73-4617-995a-9a63987da3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:46.533490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ab7dc-5b73-4617-995a-9a63987da3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:46.536164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ab7dc-5b73-4617-995a-9a63987da3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:46.536555 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ab7dc-5b73-4617-995a-9a63987da3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:46.537320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ab7dc-5b73-4617-995a-9a63987da3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:46.539011 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ab7dc-5b73-4617-995a-9a63987da3c1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:46.563197 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1364/2720] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:16:46.564927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ae56-1ab8-4393-acd5-340192c9231c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:46.576312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ae56-1ab8-4393-acd5-340192c9231c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:46.581078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ad5b8-b86e-4255-acf3-da0c95969320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:46.582324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ad5b8-b86e-4255-acf3-da0c95969320 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:46.582324 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ad5b8-b86e-4255-acf3-da0c95969320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:46.582516 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ad5b8-b86e-4255-acf3-da0c95969320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:46.583193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ad5b8-b86e-4255-acf3-da0c95969320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:46.584863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ad5b8-b86e-4255-acf3-da0c95969320 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:46.602569 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1357/2721] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:16:46.604246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ab7dc-5b73-4617-995a-9a63987da3c1 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:46.615308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ab7dc-5b73-4617-995a-9a63987da3c1 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:46.621113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c2edd-e14b-4717-9f1d-05730c7cba41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:46.622588 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c2edd-e14b-4717-9f1d-05730c7cba41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:46.622973 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c2edd-e14b-4717-9f1d-05730c7cba41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:46.623349 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c2edd-e14b-4717-9f1d-05730c7cba41 admin admin] PATH_INFO: `/v3/projects/074559fdb7be49b09dbbf66db7a1ecf4/users/126cd2f302ec4383b32ee90e0850ebb1/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:46.644099 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c2edd-e14b-4717-9f1d-05730c7cba41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=074559fdb7be49b09dbbf66db7a1ecf4, user_id=126cd2f302ec4383b32ee90e0850ebb1, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:46.648432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c2edd-e14b-4717-9f1d-05730c7cba41 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:46.666528 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1365/2722] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:16:46 2022] PUT /identity/v3/projects/074559fdb7be49b09dbbf66db7a1ecf4/users/126cd2f302ec4383b32ee90e0850ebb1/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:16:46.678169 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fa925-3337-493f-8018-1dae32733162 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:46.678720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fa925-3337-493f-8018-1dae32733162 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:46.678720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fa925-3337-493f-8018-1dae32733162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:46.712056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f23fa925-3337-493f-8018-1dae32733162 None None] MFA Rules not processed for user `126cd2f302ec4383b32ee90e0850ebb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:16:46.713464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f23fa925-3337-493f-8018-1dae32733162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:46.819379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1358/2723] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:16:46.837915 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c2edd-e14b-4717-9f1d-05730c7cba41 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:46.843713 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d60c2edd-e14b-4717-9f1d-05730c7cba41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:46.870824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fa925-3337-493f-8018-1dae32733162 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:46.872274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f23fa925-3337-493f-8018-1dae32733162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:46.888743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:46.921306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c2edd-e14b-4717-9f1d-05730c7cba41 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:46.925137 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f3b1d-7774-4d84-a3fd-443c90059dd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:46.925945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f3b1d-7774-4d84-a3fd-443c90059dd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:46.926305 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f3b1d-7774-4d84-a3fd-443c90059dd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:46.926586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f3b1d-7774-4d84-a3fd-443c90059dd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:46.935891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f3b1d-7774-4d84-a3fd-443c90059dd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:46.937348 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f3b1d-7774-4d84-a3fd-443c90059dd5 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:46.941351 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fa925-3337-493f-8018-1dae32733162 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:46.946477 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1366/2724] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:16:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:16:46.952082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:46.953159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:46.953443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:46.953812 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] PATH_INFO: `/v3/users/f7c009e1bcd74151ad3fa95c68916329` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:46.955177 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7c009e1bcd74151ad3fa95c68916329)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:46.957360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:47.010372 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7c009e1bcd74151ad3fa95c68916329'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.011104 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7c009e1bcd74151ad3fa95c68916329'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.019959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7c009e1bcd74151ad3fa95c68916329'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.024897 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:47.044035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7c009e1bcd74151ad3fa95c68916329'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.044993 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7c009e1bcd74151ad3fa95c68916329'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.059114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1359/2725] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:16:46 2022] DELETE /identity/v3/users/f7c009e1bcd74151ad3fa95c68916329 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:47.245089 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f3b1d-7774-4d84-a3fd-443c90059dd5 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:47.265123 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f3b1d-7774-4d84-a3fd-443c90059dd5 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:47.337164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:47.338398 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:47.338742 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:47.339021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] PATH_INFO: `/v3/projects/f8d7bb97464d4807a6b7bbac9a41dc67` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:47.340425 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8d7bb97464d4807a6b7bbac9a41dc67)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:47.342447 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:47.391687 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8d7bb97464d4807a6b7bbac9a41dc67', 'request_id': 'req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.397100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8d7bb97464d4807a6b7bbac9a41dc67', 'request_id': 'req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.405110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8d7bb97464d4807a6b7bbac9a41dc67', 'request_id': 'req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.421090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] The token cache is being invalidate because project f8d7bb97464d4807a6b7bbac9a41dc67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:16:47.421377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.422086 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.423764 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1367/2726] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:16:47 2022] DELETE /identity/v3/projects/f8d7bb97464d4807a6b7bbac9a41dc67 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:47.427823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:47.429318 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:47.443662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a85b3-f950-4318-9cad-0ae8bb391ab0 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:47.499878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:47.500795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:47.501072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:47.501340 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] PATH_INFO: `/v3/users/35af782693ed437d88f25d272b9be9e7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:47.502590 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35af782693ed437d88f25d272b9be9e7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:47.504538 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:47.566651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35af782693ed437d88f25d272b9be9e7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.567347 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35af782693ed437d88f25d272b9be9e7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.578042 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35af782693ed437d88f25d272b9be9e7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.594586 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35af782693ed437d88f25d272b9be9e7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.595527 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35af782693ed437d88f25d272b9be9e7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.612057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1360/2727] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:16:47 2022] DELETE /identity/v3/users/35af782693ed437d88f25d272b9be9e7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:47.811552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:47.813041 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:47.826961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a592024-9ef8-4cb6-87f6-342aab2b1a5f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:47.880430 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:47.887409 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:47.888304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:47.888729 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:47.889380 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] PATH_INFO: `/v3/projects/1438c8b530734ec5871e8a267fd2baf5` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:47.890919 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1438c8b530734ec5871e8a267fd2baf5)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:47.893179 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:47.946630 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1438c8b530734ec5871e8a267fd2baf5', 'request_id': 'req-3dfca27f-b59d-41ef-b22e-5d277513785e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.953320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1438c8b530734ec5871e8a267fd2baf5', 'request_id': 'req-3dfca27f-b59d-41ef-b22e-5d277513785e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.960574 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1438c8b530734ec5871e8a267fd2baf5', 'request_id': 'req-3dfca27f-b59d-41ef-b22e-5d277513785e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.973320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] The token cache is being invalidate because project 1438c8b530734ec5871e8a267fd2baf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:16:47.973674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.974385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:16:47.975996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1368/2728] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:16:47 2022] DELETE /identity/v3/projects/1438c8b530734ec5871e8a267fd2baf5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:16:48.823142 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:16:48.824730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:48.884583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be6476-00cd-489a-ad27-61ac77f46674 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:16:48.892994 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:48.939705 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd22757-723f-4599-a3d4-202623af761c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:16:48.940387 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd22757-723f-4599-a3d4-202623af761c service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:16:48.940701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd22757-723f-4599-a3d4-202623af761c service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:16:48.940942 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd22757-723f-4599-a3d4-202623af761c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:16:48.943113 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fbd22757-723f-4599-a3d4-202623af761c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:16:49.014959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd22757-723f-4599-a3d4-202623af761c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:16:49.016433 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd22757-723f-4599-a3d4-202623af761c service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:16:49.026866 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1361/2729] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:16:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:16:57.970334 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:16:59.023605 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=45 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:02.456134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:02.459735 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:02.531850 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfca27f-b59d-41ef-b22e-5d277513785e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:02.537197 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:02.538320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:02.538621 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:02.538946 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] PATH_INFO: `/v3/users/126cd2f302ec4383b32ee90e0850ebb1` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:02.540253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=126cd2f302ec4383b32ee90e0850ebb1)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:02.542273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:02.601419 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126cd2f302ec4383b32ee90e0850ebb1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:02.602061 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126cd2f302ec4383b32ee90e0850ebb1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:02.611707 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126cd2f302ec4383b32ee90e0850ebb1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:02.631755 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126cd2f302ec4383b32ee90e0850ebb1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:02.632706 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126cd2f302ec4383b32ee90e0850ebb1'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:02.650439 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1369/2730] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:17:02 2022] DELETE /identity/v3/users/126cd2f302ec4383b32ee90e0850ebb1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:02.652127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a6917-b361-440f-9018-ebcf3d41cabc None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:02.652609 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a6917-b361-440f-9018-ebcf3d41cabc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:02.652749 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a6917-b361-440f-9018-ebcf3d41cabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:02.671891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-368a6917-b361-440f-9018-ebcf3d41cabc None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:02.673480 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-368a6917-b361-440f-9018-ebcf3d41cabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:02.741343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1362/2731] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:02.750934 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:02.752500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:02.810489 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61cad3-f01b-4f9c-a928-fac249773977 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:02.814408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a826644-099d-4297-b023-6f3fdd389e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:02.815107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a826644-099d-4297-b023-6f3fdd389e53 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:02.815424 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a826644-099d-4297-b023-6f3fdd389e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:02.815718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a826644-099d-4297-b023-6f3fdd389e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:02.824829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a826644-099d-4297-b023-6f3fdd389e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:02.826305 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a826644-099d-4297-b023-6f3fdd389e53 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:02.836532 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1370/2732] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:17:03.131081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a6917-b361-440f-9018-ebcf3d41cabc None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:03.132591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-368a6917-b361-440f-9018-ebcf3d41cabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:03.145362 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a6917-b361-440f-9018-ebcf3d41cabc None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:03.151051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:03.151937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:03.152254 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:03.152626 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949 admin admin] PATH_INFO: `/v3/projects/074559fdb7be49b09dbbf66db7a1ecf4` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:03.153891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=074559fdb7be49b09dbbf66db7a1ecf4)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:03.156055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:03.204904 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '074559fdb7be49b09dbbf66db7a1ecf4', 'request_id': 'req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:03.211427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '074559fdb7be49b09dbbf66db7a1ecf4', 'request_id': 'req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:03.217004 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '074559fdb7be49b09dbbf66db7a1ecf4', 'request_id': 'req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:03.231686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949 admin admin] The token cache is being invalidate because project 074559fdb7be49b09dbbf66db7a1ecf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:17:03.232001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:03.232909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d5e1f5b1-8710-4ec9-b7de-58c3b4e73949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:03.235310 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1363/2733] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:17:03 2022] DELETE /identity/v3/projects/074559fdb7be49b09dbbf66db7a1ecf4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:03.243709 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef7f69-cb3d-46c9-a614-813b79e73668 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:03.244119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef7f69-cb3d-46c9-a614-813b79e73668 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:03.244552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef7f69-cb3d-46c9-a614-813b79e73668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:03.265275 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2aef7f69-cb3d-46c9-a614-813b79e73668 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:03.266542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2aef7f69-cb3d-46c9-a614-813b79e73668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:03.285175 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:03.333055 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1371/2734] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:03.398526 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4754a-8dda-4129-bcb0-347306465391 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:03.398818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4754a-8dda-4129-bcb0-347306465391 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:03.399196 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4754a-8dda-4129-bcb0-347306465391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:03.425314 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e3d4754a-8dda-4129-bcb0-347306465391 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:03.426589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e3d4754a-8dda-4129-bcb0-347306465391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:03.435029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1364/2735] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:03.439527 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef7f69-cb3d-46c9-a614-813b79e73668 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:03.450320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef7f69-cb3d-46c9-a614-813b79e73668 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:03.456595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90790aa5-82ed-4bd4-84ca-95ec5d1b2c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:03.457264 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90790aa5-82ed-4bd4-84ca-95ec5d1b2c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:03.457550 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90790aa5-82ed-4bd4-84ca-95ec5d1b2c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:03.457854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90790aa5-82ed-4bd4-84ca-95ec5d1b2c39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:03.458586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90790aa5-82ed-4bd4-84ca-95ec5d1b2c39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:17:03.458895 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90790aa5-82ed-4bd4-84ca-95ec5d1b2c39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:03.460522 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90790aa5-82ed-4bd4-84ca-95ec5d1b2c39 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:03.474413 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1372/2736] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:17:03.478738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4754a-8dda-4129-bcb0-347306465391 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:03.488139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4754a-8dda-4129-bcb0-347306465391 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:03.492960 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91c38a-e723-4e6e-a3d7-cb152b480d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:03.493612 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91c38a-e723-4e6e-a3d7-cb152b480d9e admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:03.493924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91c38a-e723-4e6e-a3d7-cb152b480d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:03.494205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91c38a-e723-4e6e-a3d7-cb152b480d9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:03.494901 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91c38a-e723-4e6e-a3d7-cb152b480d9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:03.496811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91c38a-e723-4e6e-a3d7-cb152b480d9e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:03.528535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1365/2737] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:17:03 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:03.535113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90790aa5-82ed-4bd4-84ca-95ec5d1b2c39 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:03.550202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90790aa5-82ed-4bd4-84ca-95ec5d1b2c39 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:03.610834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e90d8-5fcb-4c85-b1e6-c25843e8e083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:03.611501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e90d8-5fcb-4c85-b1e6-c25843e8e083 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:03.611979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e90d8-5fcb-4c85-b1e6-c25843e8e083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:03.612269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e90d8-5fcb-4c85-b1e6-c25843e8e083 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:03.613057 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e90d8-5fcb-4c85-b1e6-c25843e8e083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:03.615045 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e90d8-5fcb-4c85-b1e6-c25843e8e083 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:03.638705 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1373/2738] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:17:03 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:03.643401 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91c38a-e723-4e6e-a3d7-cb152b480d9e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:03.654501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91c38a-e723-4e6e-a3d7-cb152b480d9e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:03.659392 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0a47d-0af0-4ba5-b726-bc1653a29466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:03.660170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0a47d-0af0-4ba5-b726-bc1653a29466 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:03.660483 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0a47d-0af0-4ba5-b726-bc1653a29466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:03.660916 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0a47d-0af0-4ba5-b726-bc1653a29466 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:03.661804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0a47d-0af0-4ba5-b726-bc1653a29466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:03.663440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0a47d-0af0-4ba5-b726-bc1653a29466 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:03.678051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1366/2739] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:03.682716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e90d8-5fcb-4c85-b1e6-c25843e8e083 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:03.693654 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e90d8-5fcb-4c85-b1e6-c25843e8e083 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:03.698721 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce7f16-59c5-4ef9-b021-2ccca07c6450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:03.699374 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce7f16-59c5-4ef9-b021-2ccca07c6450 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:03.699662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce7f16-59c5-4ef9-b021-2ccca07c6450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:03.699945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce7f16-59c5-4ef9-b021-2ccca07c6450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:03.700671 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce7f16-59c5-4ef9-b021-2ccca07c6450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:03.702347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce7f16-59c5-4ef9-b021-2ccca07c6450 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:03.713066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1374/2740] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:03.717213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0a47d-0af0-4ba5-b726-bc1653a29466 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:03.728014 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0a47d-0af0-4ba5-b726-bc1653a29466 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:03.733660 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35408887-e3c4-4976-8a59-c7a347bdc253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:03.734548 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35408887-e3c4-4976-8a59-c7a347bdc253 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:03.734831 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35408887-e3c4-4976-8a59-c7a347bdc253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:03.735118 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35408887-e3c4-4976-8a59-c7a347bdc253 admin admin] PATH_INFO: `/v3/projects/aaff1fa03a194074aa14fae9db94cefb/users/2321028b4d4f4734afd1ce674931a4d3/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:03.750863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35408887-e3c4-4976-8a59-c7a347bdc253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaff1fa03a194074aa14fae9db94cefb, user_id=2321028b4d4f4734afd1ce674931a4d3, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:03.754076 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35408887-e3c4-4976-8a59-c7a347bdc253 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:03.768452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1367/2741] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:03 2022] PUT /identity/v3/projects/aaff1fa03a194074aa14fae9db94cefb/users/2321028b4d4f4734afd1ce674931a4d3/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:03.773817 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4458f6-10c7-441c-b66d-26a8b6c3c7fd None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:03.774132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4458f6-10c7-441c-b66d-26a8b6c3c7fd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:03.774410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4458f6-10c7-441c-b66d-26a8b6c3c7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:03.793563 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1e4458f6-10c7-441c-b66d-26a8b6c3c7fd None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:03.795096 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1e4458f6-10c7-441c-b66d-26a8b6c3c7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:03.862138 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1375/2742] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:03.872215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35408887-e3c4-4976-8a59-c7a347bdc253 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:03.874360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-35408887-e3c4-4976-8a59-c7a347bdc253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:03.897556 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:03.938088 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35408887-e3c4-4976-8a59-c7a347bdc253 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:03.942213 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b151b7-e105-4fac-a8c6-c14174bab925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:03.942872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b151b7-e105-4fac-a8c6-c14174bab925 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:03.943159 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b151b7-e105-4fac-a8c6-c14174bab925 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:03.943443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b151b7-e105-4fac-a8c6-c14174bab925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:03.953109 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b151b7-e105-4fac-a8c6-c14174bab925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:03.954677 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b151b7-e105-4fac-a8c6-c14174bab925 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:03.965346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1368/2743] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:17:08.400015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5cff8-0218-45d9-99c2-cdf3a218db04 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:08.400515 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5cff8-0218-45d9-99c2-cdf3a218db04 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:08.400740 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5cff8-0218-45d9-99c2-cdf3a218db04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:08.430521 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2ee5cff8-0218-45d9-99c2-cdf3a218db04 None None] MFA Rules not processed for user `2321028b4d4f4734afd1ce674931a4d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:08.431972 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2ee5cff8-0218-45d9-99c2-cdf3a218db04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:08.458215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:08.517433 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1376/2744] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:08.522078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b151b7-e105-4fac-a8c6-c14174bab925 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:08.532497 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b151b7-e105-4fac-a8c6-c14174bab925 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:08.537270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80039e9d-70ff-49dd-b830-a5560d457211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:08.537962 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80039e9d-70ff-49dd-b830-a5560d457211 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:08.538243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80039e9d-70ff-49dd-b830-a5560d457211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:08.538514 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80039e9d-70ff-49dd-b830-a5560d457211 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:08.539209 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80039e9d-70ff-49dd-b830-a5560d457211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:08.541123 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80039e9d-70ff-49dd-b830-a5560d457211 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:08.569327 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1369/2745] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:17:08 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:08.573488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5cff8-0218-45d9-99c2-cdf3a218db04 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:08.586365 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5cff8-0218-45d9-99c2-cdf3a218db04 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:08.645028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa1ac6-7784-4785-9d3f-3f40f2ed8c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:08.645726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa1ac6-7784-4785-9d3f-3f40f2ed8c66 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:08.646008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa1ac6-7784-4785-9d3f-3f40f2ed8c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:08.646278 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa1ac6-7784-4785-9d3f-3f40f2ed8c66 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:08.646954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa1ac6-7784-4785-9d3f-3f40f2ed8c66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:08.648891 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa1ac6-7784-4785-9d3f-3f40f2ed8c66 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:08.672617 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1377/2746] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:17:08 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:08.677348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80039e9d-70ff-49dd-b830-a5560d457211 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:08.688103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80039e9d-70ff-49dd-b830-a5560d457211 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:08.693060 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107346ed-fd36-4e69-8c6d-4f57780b75dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:08.693779 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107346ed-fd36-4e69-8c6d-4f57780b75dd admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:08.694061 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107346ed-fd36-4e69-8c6d-4f57780b75dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:08.694336 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107346ed-fd36-4e69-8c6d-4f57780b75dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:08.695083 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107346ed-fd36-4e69-8c6d-4f57780b75dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:08.696678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107346ed-fd36-4e69-8c6d-4f57780b75dd admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:08.707909 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1370/2747] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:08.712594 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa1ac6-7784-4785-9d3f-3f40f2ed8c66 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:08.723110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa1ac6-7784-4785-9d3f-3f40f2ed8c66 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:08.728209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbdee0-7149-453f-90ac-14f0907e6f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:08.729109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbdee0-7149-453f-90ac-14f0907e6f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:08.729383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbdee0-7149-453f-90ac-14f0907e6f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:08.729689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbdee0-7149-453f-90ac-14f0907e6f8c admin admin] PATH_INFO: `/v3/projects/69bd956122ce4cc8b7d4d97f4f0c1868/users/ead485491f7d4c4b85e2974d377996c2/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:08.745765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbdee0-7149-453f-90ac-14f0907e6f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=69bd956122ce4cc8b7d4d97f4f0c1868, user_id=ead485491f7d4c4b85e2974d377996c2, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:08.748499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbdee0-7149-453f-90ac-14f0907e6f8c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:08.765085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1378/2748] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:08 2022] PUT /identity/v3/projects/69bd956122ce4cc8b7d4d97f4f0c1868/users/ead485491f7d4c4b85e2974d377996c2/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:08.769908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107346ed-fd36-4e69-8c6d-4f57780b75dd admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:08.784188 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107346ed-fd36-4e69-8c6d-4f57780b75dd admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:08.843133 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642618a2-8b83-4799-8bee-e4b6ea76390c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:08.843803 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642618a2-8b83-4799-8bee-e4b6ea76390c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:08.844081 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642618a2-8b83-4799-8bee-e4b6ea76390c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:08.844352 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642618a2-8b83-4799-8bee-e4b6ea76390c admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:08.845070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642618a2-8b83-4799-8bee-e4b6ea76390c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:08.846858 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642618a2-8b83-4799-8bee-e4b6ea76390c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:08.857114 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1371/2749] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:08.862722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbdee0-7149-453f-90ac-14f0907e6f8c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:08.874056 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbdee0-7149-453f-90ac-14f0907e6f8c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:08.879542 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417047ab-8a61-4cff-984f-c12e3001296b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:08.880441 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417047ab-8a61-4cff-984f-c12e3001296b admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:08.880727 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417047ab-8a61-4cff-984f-c12e3001296b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:08.881007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417047ab-8a61-4cff-984f-c12e3001296b admin admin] PATH_INFO: `/v3/projects/69bd956122ce4cc8b7d4d97f4f0c1868/users/ead485491f7d4c4b85e2974d377996c2/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:08.883103 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417047ab-8a61-4cff-984f-c12e3001296b admin admin] RBAC: Authorizing `identity:create_grant(project_id=69bd956122ce4cc8b7d4d97f4f0c1868, user_id=ead485491f7d4c4b85e2974d377996c2, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:08.885816 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417047ab-8a61-4cff-984f-c12e3001296b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:08.896492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1379/2750] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:08 2022] PUT /identity/v3/projects/69bd956122ce4cc8b7d4d97f4f0c1868/users/ead485491f7d4c4b85e2974d377996c2/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:13.694693 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e6ec2-2a3c-4a28-a838-6b5138424ab0 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:13.695239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e6ec2-2a3c-4a28-a838-6b5138424ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:13.695239 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e6ec2-2a3c-4a28-a838-6b5138424ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:13.731855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-140e6ec2-2a3c-4a28-a838-6b5138424ab0 None None] MFA Rules not processed for user `ead485491f7d4c4b85e2974d377996c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:13.733086 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-140e6ec2-2a3c-4a28-a838-6b5138424ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:13.814589 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1372/2751] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:17:13.833593 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417047ab-8a61-4cff-984f-c12e3001296b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:13.836088 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-417047ab-8a61-4cff-984f-c12e3001296b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:13.918195 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417047ab-8a61-4cff-984f-c12e3001296b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:13.923993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306754ed-b6b5-455b-85bf-5f85e8a170fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:13.924722 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306754ed-b6b5-455b-85bf-5f85e8a170fe service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:13.924998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306754ed-b6b5-455b-85bf-5f85e8a170fe service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:13.925446 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306754ed-b6b5-455b-85bf-5f85e8a170fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:13.932531 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:13.997454 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306754ed-b6b5-455b-85bf-5f85e8a170fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:13.999198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306754ed-b6b5-455b-85bf-5f85e8a170fe service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:14.009467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1380/2752] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:17:14.747052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e6ec2-2a3c-4a28-a838-6b5138424ab0 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:14.748651 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-140e6ec2-2a3c-4a28-a838-6b5138424ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:14.763074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:14.821163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e6ec2-2a3c-4a28-a838-6b5138424ab0 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:14.826802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:14.827714 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:14.828000 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:14.828290 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] PATH_INFO: `/v3/users/da29008234dc444a8fd99cd5a5bcc4cc` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:14.829510 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] RBAC: Authorizing `identity:delete_user(user_id=da29008234dc444a8fd99cd5a5bcc4cc)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:14.831578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:14.893255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da29008234dc444a8fd99cd5a5bcc4cc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:14.894440 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da29008234dc444a8fd99cd5a5bcc4cc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:14.903183 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da29008234dc444a8fd99cd5a5bcc4cc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:14.916958 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da29008234dc444a8fd99cd5a5bcc4cc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:14.917665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da29008234dc444a8fd99cd5a5bcc4cc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:14.930922 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1373/2753] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:17:14 2022] DELETE /identity/v3/users/da29008234dc444a8fd99cd5a5bcc4cc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:14.935608 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a1ba7-62e8-45b6-b8e2-19a56c383348 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:14.936085 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a1ba7-62e8-45b6-b8e2-19a56c383348 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:14.936192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a1ba7-62e8-45b6-b8e2-19a56c383348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:14.953423 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6a4a1ba7-62e8-45b6-b8e2-19a56c383348 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:14.954726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6a4a1ba7-62e8-45b6-b8e2-19a56c383348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:15.025339 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1381/2754] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:15.035872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:15.037337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:15.129022 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68562a42-90c4-4384-b57f-06792151179d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:15.138410 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cd1b2-e370-433b-9b4c-8a2e8b522ab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:15.142566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cd1b2-e370-433b-9b4c-8a2e8b522ab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:15.144741 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cd1b2-e370-433b-9b4c-8a2e8b522ab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:15.145234 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cd1b2-e370-433b-9b4c-8a2e8b522ab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:15.160505 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cd1b2-e370-433b-9b4c-8a2e8b522ab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:15.162136 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cd1b2-e370-433b-9b4c-8a2e8b522ab8 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:15.177840 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1374/2755] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:17:15.515177 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a1ba7-62e8-45b6-b8e2-19a56c383348 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:15.516956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6a4a1ba7-62e8-45b6-b8e2-19a56c383348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:15.532214 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a1ba7-62e8-45b6-b8e2-19a56c383348 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:15.540019 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:15.541101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:15.541485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:15.541917 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] PATH_INFO: `/v3/projects/45dd96c90c824c45afd3b6b345b47e7d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:15.543280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45dd96c90c824c45afd3b6b345b47e7d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:15.545589 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:15.607751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45dd96c90c824c45afd3b6b345b47e7d', 'request_id': 'req-b6c35d1d-6c98-420a-9c8c-762853785d48'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:15.614180 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45dd96c90c824c45afd3b6b345b47e7d', 'request_id': 'req-b6c35d1d-6c98-420a-9c8c-762853785d48'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:15.620949 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45dd96c90c824c45afd3b6b345b47e7d', 'request_id': 'req-b6c35d1d-6c98-420a-9c8c-762853785d48'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:15.635290 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] The token cache is being invalidate because project 45dd96c90c824c45afd3b6b345b47e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:17:15.635586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:15.636236 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:15.640638 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1382/2756] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:17:15 2022] DELETE /identity/v3/projects/45dd96c90c824c45afd3b6b345b47e7d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:15.642168 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cd1b2-e370-433b-9b4c-8a2e8b522ab8 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:15.653129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cd1b2-e370-433b-9b4c-8a2e8b522ab8 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:15.713110 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:15.760922 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ecad4-8e6f-4a62-89d0-026e5411d1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:15.762398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ecad4-8e6f-4a62-89d0-026e5411d1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:15.762759 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ecad4-8e6f-4a62-89d0-026e5411d1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:15.763621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ecad4-8e6f-4a62-89d0-026e5411d1b0 admin admin] PATH_INFO: `/v3/users/f3019ab9f06143b1aa0391fd80babd06` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:15.765127 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ecad4-8e6f-4a62-89d0-026e5411d1b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3019ab9f06143b1aa0391fd80babd06)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:15.767255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ecad4-8e6f-4a62-89d0-026e5411d1b0 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:15.820402 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d37ecad4-8e6f-4a62-89d0-026e5411d1b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3019ab9f06143b1aa0391fd80babd06'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:15.821051 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d37ecad4-8e6f-4a62-89d0-026e5411d1b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3019ab9f06143b1aa0391fd80babd06'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:15.829606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d37ecad4-8e6f-4a62-89d0-026e5411d1b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3019ab9f06143b1aa0391fd80babd06'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:15.843678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d37ecad4-8e6f-4a62-89d0-026e5411d1b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3019ab9f06143b1aa0391fd80babd06'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:15.844421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d37ecad4-8e6f-4a62-89d0-026e5411d1b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3019ab9f06143b1aa0391fd80babd06'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:15.857655 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1375/2757] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:17:15 2022] DELETE /identity/v3/users/f3019ab9f06143b1aa0391fd80babd06 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:16.192653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:16.194174 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:16.209206 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c35d1d-6c98-420a-9c8c-762853785d48 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:16.242253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:16.276645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c312598-feb0-48c8-a08c-a2c96b7e5eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:16.277540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312598-feb0-48c8-a08c-a2c96b7e5eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:16.277852 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312598-feb0-48c8-a08c-a2c96b7e5eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:16.278125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312598-feb0-48c8-a08c-a2c96b7e5eca admin admin] PATH_INFO: `/v3/projects/796a3326c4d74dc68a84cd6fceae7e2f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:16.279336 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c312598-feb0-48c8-a08c-a2c96b7e5eca admin admin] RBAC: Authorizing `identity:delete_project(project_id=796a3326c4d74dc68a84cd6fceae7e2f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:16.281324 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c312598-feb0-48c8-a08c-a2c96b7e5eca admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:16.332786 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9c312598-feb0-48c8-a08c-a2c96b7e5eca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '796a3326c4d74dc68a84cd6fceae7e2f', 'request_id': 'req-9c312598-feb0-48c8-a08c-a2c96b7e5eca'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:16.343932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9c312598-feb0-48c8-a08c-a2c96b7e5eca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '796a3326c4d74dc68a84cd6fceae7e2f', 'request_id': 'req-9c312598-feb0-48c8-a08c-a2c96b7e5eca'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:16.351400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9c312598-feb0-48c8-a08c-a2c96b7e5eca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '796a3326c4d74dc68a84cd6fceae7e2f', 'request_id': 'req-9c312598-feb0-48c8-a08c-a2c96b7e5eca'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:16.365210 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9c312598-feb0-48c8-a08c-a2c96b7e5eca admin admin] The token cache is being invalidate because project 796a3326c4d74dc68a84cd6fceae7e2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:17:16.365499 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9c312598-feb0-48c8-a08c-a2c96b7e5eca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:16.366273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9c312598-feb0-48c8-a08c-a2c96b7e5eca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:16.371051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1383/2758] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:17:16 2022] DELETE /identity/v3/projects/796a3326c4d74dc68a84cd6fceae7e2f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:16.372782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0464e-bac3-4685-9fbe-ce7517399a99 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:16.373074 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0464e-bac3-4685-9fbe-ce7517399a99 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:16.373360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0464e-bac3-4685-9fbe-ce7517399a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:16.396833 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1cf0464e-bac3-4685-9fbe-ce7517399a99 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:16.398730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1cf0464e-bac3-4685-9fbe-ce7517399a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:16.483812 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06369024-bd9a-4328-b4fa-5159f8bdb009 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:16.484132 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06369024-bd9a-4328-b4fa-5159f8bdb009 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:16.484415 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06369024-bd9a-4328-b4fa-5159f8bdb009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:16.485350 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1376/2759] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:16.504212 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-06369024-bd9a-4328-b4fa-5159f8bdb009 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:16.505773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-06369024-bd9a-4328-b4fa-5159f8bdb009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:16.514359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1384/2760] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:17:16.519001 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0464e-bac3-4685-9fbe-ce7517399a99 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:16.529160 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0464e-bac3-4685-9fbe-ce7517399a99 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:16.535454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f811e94-63bd-44f8-ace6-4f5c37f74955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:16.536263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f811e94-63bd-44f8-ace6-4f5c37f74955 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:16.536613 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f811e94-63bd-44f8-ace6-4f5c37f74955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:16.536957 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f811e94-63bd-44f8-ace6-4f5c37f74955 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:16.537823 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f811e94-63bd-44f8-ace6-4f5c37f74955 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:17:16.538203 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f811e94-63bd-44f8-ace6-4f5c37f74955 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:16.539943 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f811e94-63bd-44f8-ace6-4f5c37f74955 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:16.555339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1377/2761] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:17:16.559712 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06369024-bd9a-4328-b4fa-5159f8bdb009 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:16.570451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06369024-bd9a-4328-b4fa-5159f8bdb009 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:16.575760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5208a3-3ffc-41b5-a6ea-483b55c5eb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:16.576463 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5208a3-3ffc-41b5-a6ea-483b55c5eb55 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:16.576773 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5208a3-3ffc-41b5-a6ea-483b55c5eb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:16.577044 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5208a3-3ffc-41b5-a6ea-483b55c5eb55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:16.577762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5208a3-3ffc-41b5-a6ea-483b55c5eb55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:16.579679 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5208a3-3ffc-41b5-a6ea-483b55c5eb55 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:16.613076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1385/2762] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:17:16 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:16.617194 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f811e94-63bd-44f8-ace6-4f5c37f74955 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:16.632036 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f811e94-63bd-44f8-ace6-4f5c37f74955 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:16.696821 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:16.730537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d90fdc-e657-4715-bfbb-50dfa883becf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:16.732641 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d90fdc-e657-4715-bfbb-50dfa883becf admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:16.733035 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d90fdc-e657-4715-bfbb-50dfa883becf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:16.733322 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d90fdc-e657-4715-bfbb-50dfa883becf admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:16.734281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d90fdc-e657-4715-bfbb-50dfa883becf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:16.736294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d90fdc-e657-4715-bfbb-50dfa883becf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:16.780587 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1378/2763] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:17:16 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:17:16.785033 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5208a3-3ffc-41b5-a6ea-483b55c5eb55 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:16.800840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5208a3-3ffc-41b5-a6ea-483b55c5eb55 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:16.808389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ed333-12ab-480e-8f6a-59b63d25293c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:16.809093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ed333-12ab-480e-8f6a-59b63d25293c admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:16.809377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ed333-12ab-480e-8f6a-59b63d25293c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:16.809745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ed333-12ab-480e-8f6a-59b63d25293c admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:16.810428 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ed333-12ab-480e-8f6a-59b63d25293c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:16.812131 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ed333-12ab-480e-8f6a-59b63d25293c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:16.828575 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1386/2764] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:16.830102 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d90fdc-e657-4715-bfbb-50dfa883becf admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:16.852216 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d90fdc-e657-4715-bfbb-50dfa883becf admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:16.857237 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a2efc-83f9-42d9-bca1-3d44802b7042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:16.857984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a2efc-83f9-42d9-bca1-3d44802b7042 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:16.858277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a2efc-83f9-42d9-bca1-3d44802b7042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:16.858561 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a2efc-83f9-42d9-bca1-3d44802b7042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:16.859310 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a2efc-83f9-42d9-bca1-3d44802b7042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:16.860997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a2efc-83f9-42d9-bca1-3d44802b7042 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:16.875635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1379/2765] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:17:16.879682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ed333-12ab-480e-8f6a-59b63d25293c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:16.892484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ed333-12ab-480e-8f6a-59b63d25293c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:16.897566 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe66ca-b10c-4c28-81e4-66eea8083313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:16.898760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe66ca-b10c-4c28-81e4-66eea8083313 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:16.899042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe66ca-b10c-4c28-81e4-66eea8083313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:16.899315 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe66ca-b10c-4c28-81e4-66eea8083313 admin admin] PATH_INFO: `/v3/projects/65654bd6e75542a093b478139cadf49b/users/bf2f5101f6554643bdcaa21b194377ec/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:16.918198 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe66ca-b10c-4c28-81e4-66eea8083313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65654bd6e75542a093b478139cadf49b, user_id=bf2f5101f6554643bdcaa21b194377ec, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:16.920911 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe66ca-b10c-4c28-81e4-66eea8083313 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:16.938536 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1387/2766] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:16 2022] PUT /identity/v3/projects/65654bd6e75542a093b478139cadf49b/users/bf2f5101f6554643bdcaa21b194377ec/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:16.955263 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96520855-6be1-49f6-9c97-960205924ebf None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:16.955717 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96520855-6be1-49f6-9c97-960205924ebf None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:16.956108 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96520855-6be1-49f6-9c97-960205924ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:16.999382 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-96520855-6be1-49f6-9c97-960205924ebf None None] MFA Rules not processed for user `bf2f5101f6554643bdcaa21b194377ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:17.000606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-96520855-6be1-49f6-9c97-960205924ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:17.078398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1380/2767] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:17.083551 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe66ca-b10c-4c28-81e4-66eea8083313 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:17.168960 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:17.178067 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe66ca-b10c-4c28-81e4-66eea8083313 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:17.182342 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe17c48-b501-476d-ade7-ff01c72752df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:17.182754 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe17c48-b501-476d-ade7-ff01c72752df admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:17.183356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe17c48-b501-476d-ade7-ff01c72752df admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:17.183651 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe17c48-b501-476d-ade7-ff01c72752df admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:17.184472 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe17c48-b501-476d-ade7-ff01c72752df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:17.187134 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe17c48-b501-476d-ade7-ff01c72752df admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:17.239979 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1388/2768] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:17:17 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:17.244498 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96520855-6be1-49f6-9c97-960205924ebf None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:17.257275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96520855-6be1-49f6-9c97-960205924ebf None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:17.315080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe17c48-b501-476d-ade7-ff01c72752df admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:17.317400 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7fe17c48-b501-476d-ade7-ff01c72752df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:17.347459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe3a85-4cc1-4bc4-9b77-bc3b0757257c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:17.348165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe3a85-4cc1-4bc4-9b77-bc3b0757257c admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:17.348456 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe3a85-4cc1-4bc4-9b77-bc3b0757257c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:17.348832 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe3a85-4cc1-4bc4-9b77-bc3b0757257c admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:17.349540 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe3a85-4cc1-4bc4-9b77-bc3b0757257c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:17.352679 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe3a85-4cc1-4bc4-9b77-bc3b0757257c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:17.379653 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1381/2769] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:17:17 2022] POST /identity/v3/users => generated 297 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:17.384112 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe3a85-4cc1-4bc4-9b77-bc3b0757257c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:17.394686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe3a85-4cc1-4bc4-9b77-bc3b0757257c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:17.399575 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72764899-5924-47ce-ba3b-279083c423de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:17.400407 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72764899-5924-47ce-ba3b-279083c423de admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:17.400813 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72764899-5924-47ce-ba3b-279083c423de admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:17.401591 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72764899-5924-47ce-ba3b-279083c423de admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:17.401846 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72764899-5924-47ce-ba3b-279083c423de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:17.403466 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72764899-5924-47ce-ba3b-279083c423de admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:17.421144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1382/2770] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:17:17.425807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72764899-5924-47ce-ba3b-279083c423de admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:17.428422 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe17c48-b501-476d-ade7-ff01c72752df admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:17.436616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72764899-5924-47ce-ba3b-279083c423de admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:17.442568 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f715f12-76d9-4a33-bcf1-4a66966f2bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:17.442812 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:17.444095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f715f12-76d9-4a33-bcf1-4a66966f2bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:17.444095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f715f12-76d9-4a33-bcf1-4a66966f2bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:17.444332 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f715f12-76d9-4a33-bcf1-4a66966f2bbf admin admin] PATH_INFO: `/v3/projects/08bd36b51ea940bab7f3c9be7244eb82/users/266c618033ec4b1d8a541a1ffc6a495d/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:17.449080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:17.449435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:17.449752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] PATH_INFO: `/v3/users/2495a8406c184164982d136edc351cfd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:17.466302 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2495a8406c184164982d136edc351cfd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:17.476967 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:17.486896 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f715f12-76d9-4a33-bcf1-4a66966f2bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=08bd36b51ea940bab7f3c9be7244eb82, user_id=266c618033ec4b1d8a541a1ffc6a495d, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:17.489664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f715f12-76d9-4a33-bcf1-4a66966f2bbf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:17.517321 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1383/2771] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:17 2022] PUT /identity/v3/projects/08bd36b51ea940bab7f3c9be7244eb82/users/266c618033ec4b1d8a541a1ffc6a495d/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:17:17.522780 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f715f12-76d9-4a33-bcf1-4a66966f2bbf admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:17.531720 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:17.552498 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2495a8406c184164982d136edc351cfd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:17.553190 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2495a8406c184164982d136edc351cfd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:17.564602 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2495a8406c184164982d136edc351cfd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:17.582003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2495a8406c184164982d136edc351cfd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:17.583001 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2495a8406c184164982d136edc351cfd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:17.600834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1389/2772] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:17:17 2022] DELETE /identity/v3/users/2495a8406c184164982d136edc351cfd => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:17.610327 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f715f12-76d9-4a33-bcf1-4a66966f2bbf admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:17.677016 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b661042-c418-45e6-8e2b-9689f0ba4048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:17.677739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b661042-c418-45e6-8e2b-9689f0ba4048 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:17.678022 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b661042-c418-45e6-8e2b-9689f0ba4048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:17.678293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b661042-c418-45e6-8e2b-9689f0ba4048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:17.679018 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b661042-c418-45e6-8e2b-9689f0ba4048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:17.681490 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b661042-c418-45e6-8e2b-9689f0ba4048 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:17.693661 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1384/2773] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:17.698015 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:17.701040 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:17.720177 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59411131-4992-4997-8467-bbadaea79aa8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:17.726490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f85ba-15d5-4f50-9172-e5510e9d31af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:17.727559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f85ba-15d5-4f50-9172-e5510e9d31af admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:17.727966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f85ba-15d5-4f50-9172-e5510e9d31af admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:17.728245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f85ba-15d5-4f50-9172-e5510e9d31af admin admin] PATH_INFO: `/v3/projects/08bd36b51ea940bab7f3c9be7244eb82/users/266c618033ec4b1d8a541a1ffc6a495d/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:17.730353 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f85ba-15d5-4f50-9172-e5510e9d31af admin admin] RBAC: Authorizing `identity:create_grant(project_id=08bd36b51ea940bab7f3c9be7244eb82, user_id=266c618033ec4b1d8a541a1ffc6a495d, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:17.733110 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f85ba-15d5-4f50-9172-e5510e9d31af admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:17.744774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1390/2774] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:17 2022] PUT /identity/v3/projects/08bd36b51ea940bab7f3c9be7244eb82/users/266c618033ec4b1d8a541a1ffc6a495d/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:17:17.755439 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c846c-732e-4fcc-8653-77f0a1e76b69 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:17.755729 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c846c-732e-4fcc-8653-77f0a1e76b69 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:17.756011 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c846c-732e-4fcc-8653-77f0a1e76b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:17.792849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-600c846c-732e-4fcc-8653-77f0a1e76b69 None None] MFA Rules not processed for user `266c618033ec4b1d8a541a1ffc6a495d`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:17.794122 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-600c846c-732e-4fcc-8653-77f0a1e76b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:17.889686 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1385/2775] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:17:17.906173 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f85ba-15d5-4f50-9172-e5510e9d31af admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:17.907968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-153f85ba-15d5-4f50-9172-e5510e9d31af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:17.992328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:18.004739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f85ba-15d5-4f50-9172-e5510e9d31af admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:18.011000 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d304ff3-4632-472c-b96f-8cc90e145a66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:18.012021 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d304ff3-4632-472c-b96f-8cc90e145a66 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.012511 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d304ff3-4632-472c-b96f-8cc90e145a66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.013010 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d304ff3-4632-472c-b96f-8cc90e145a66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.026197 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d304ff3-4632-472c-b96f-8cc90e145a66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:18.027296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d304ff3-4632-472c-b96f-8cc90e145a66 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:18.039008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1391/2776] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:17:18.126429 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c846c-732e-4fcc-8653-77f0a1e76b69 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:18.128167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-600c846c-732e-4fcc-8653-77f0a1e76b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:18.184394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:18.218917 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c846c-732e-4fcc-8653-77f0a1e76b69 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:18.224562 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:18.225443 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.225762 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.226043 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb admin admin] PATH_INFO: `/v3/projects/02795b95a9b84564999233883b763e4f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.227241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=02795b95a9b84564999233883b763e4f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:18.229195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:18.278574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02795b95a9b84564999233883b763e4f', 'request_id': 'req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:18.285333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02795b95a9b84564999233883b763e4f', 'request_id': 'req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:18.291698 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02795b95a9b84564999233883b763e4f', 'request_id': 'req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:18.307044 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb admin admin] The token cache is being invalidate because project 02795b95a9b84564999233883b763e4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:17:18.307358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:18.308052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b25a1139-7a0d-44e1-9a4a-9db9bf9e1ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:18.310248 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1386/2777] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:17:18 2022] DELETE /identity/v3/projects/02795b95a9b84564999233883b763e4f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:18.315904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e97a23-307b-4f5e-a24d-efd2d92c4218 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.316452 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e97a23-307b-4f5e-a24d-efd2d92c4218 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.316452 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e97a23-307b-4f5e-a24d-efd2d92c4218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.334994 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-99e97a23-307b-4f5e-a24d-efd2d92c4218 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:18.336619 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-99e97a23-307b-4f5e-a24d-efd2d92c4218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:18.399076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1392/2778] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:17:18.407477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed7de1-a694-49b5-ac44-d0d59e371bec None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.407736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed7de1-a694-49b5-ac44-d0d59e371bec None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.408045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed7de1-a694-49b5-ac44-d0d59e371bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.417715 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e97a23-307b-4f5e-a24d-efd2d92c4218 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:18.419237 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-99e97a23-307b-4f5e-a24d-efd2d92c4218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:18.427696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e1ed7de1-a694-49b5-ac44-d0d59e371bec None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:18.428927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e1ed7de1-a694-49b5-ac44-d0d59e371bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:18.437055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1387/2779] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:18.442368 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed7de1-a694-49b5-ac44-d0d59e371bec None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:18.452477 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed7de1-a694-49b5-ac44-d0d59e371bec None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:18.458073 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c695610-8a14-4121-81fb-f352092df3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:18.458849 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c695610-8a14-4121-81fb-f352092df3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.459137 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c695610-8a14-4121-81fb-f352092df3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.459427 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c695610-8a14-4121-81fb-f352092df3e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.460171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c695610-8a14-4121-81fb-f352092df3e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:17:18.460489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c695610-8a14-4121-81fb-f352092df3e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:18.462535 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c695610-8a14-4121-81fb-f352092df3e6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:18.474732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1388/2780] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:17:18.479091 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c695610-8a14-4121-81fb-f352092df3e6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:18.489871 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e97a23-307b-4f5e-a24d-efd2d92c4218 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:18.490066 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c695610-8a14-4121-81fb-f352092df3e6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:18.494571 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff580261-8b22-4bf0-9eda-ab8bde7dcf83 tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266c618033ec4b1d8a541a1ffc6a495d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08bd36b51ea940bab7f3c9be7244eb82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:18.494750 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4d78d-3527-4545-8735-c3a731764bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:18.495235 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff580261-8b22-4bf0-9eda-ab8bde7dcf83 tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.495374 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4d78d-3527-4545-8735-c3a731764bac admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.495525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff580261-8b22-4bf0-9eda-ab8bde7dcf83 tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.495667 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4d78d-3527-4545-8735-c3a731764bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.495804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff580261-8b22-4bf0-9eda-ab8bde7dcf83 tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.495908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4d78d-3527-4545-8735-c3a731764bac admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.496525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4d78d-3527-4545-8735-c3a731764bac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:18.496645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff580261-8b22-4bf0-9eda-ab8bde7dcf83 tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:17:18.496912 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff580261-8b22-4bf0-9eda-ab8bde7dcf83 tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:18.498479 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4d78d-3527-4545-8735-c3a731764bac admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:18.498596 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff580261-8b22-4bf0-9eda-ab8bde7dcf83 tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:18.511660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1393/2781] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:17:18.516389 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff580261-8b22-4bf0-9eda-ab8bde7dcf83 tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:18.527444 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff580261-8b22-4bf0-9eda-ab8bde7dcf83 tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:18.532107 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a3d9c-3aa2-4cff-81bb-1bfb66c2ed0f tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266c618033ec4b1d8a541a1ffc6a495d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08bd36b51ea940bab7f3c9be7244eb82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:18.532772 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a3d9c-3aa2-4cff-81bb-1bfb66c2ed0f tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.533084 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a3d9c-3aa2-4cff-81bb-1bfb66c2ed0f tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.533367 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a3d9c-3aa2-4cff-81bb-1bfb66c2ed0f tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.534108 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a3d9c-3aa2-4cff-81bb-1bfb66c2ed0f tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:18.534671 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1389/2782] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:17:18 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:17:18.536023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a3d9c-3aa2-4cff-81bb-1bfb66c2ed0f tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:18.538767 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4d78d-3527-4545-8735-c3a731764bac admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:18.566320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1394/2783] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:17:18 2022] POST /identity/v3/projects => generated 354 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:17:18.567861 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a3d9c-3aa2-4cff-81bb-1bfb66c2ed0f tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:18.618688 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4d78d-3527-4545-8735-c3a731764bac admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:18.642623 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:18.648452 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a3d9c-3aa2-4cff-81bb-1bfb66c2ed0f tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:18.653849 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19a01e-4fd4-4526-80ab-58f0858a225e tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266c618033ec4b1d8a541a1ffc6a495d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08bd36b51ea940bab7f3c9be7244eb82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:18.655308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19a01e-4fd4-4526-80ab-58f0858a225e tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.655308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19a01e-4fd4-4526-80ab-58f0858a225e tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.655308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19a01e-4fd4-4526-80ab-58f0858a225e tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.655839 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19a01e-4fd4-4526-80ab-58f0858a225e tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:17:18.656152 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19a01e-4fd4-4526-80ab-58f0858a225e tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:18.657802 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19a01e-4fd4-4526-80ab-58f0858a225e tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:18.665867 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:18.673068 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1395/2784] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:17:18.688531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe3aff-b6f4-4e18-a395-92cc711e4645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:18.689215 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe3aff-b6f4-4e18-a395-92cc711e4645 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.689503 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe3aff-b6f4-4e18-a395-92cc711e4645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.690148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe3aff-b6f4-4e18-a395-92cc711e4645 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.690959 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe3aff-b6f4-4e18-a395-92cc711e4645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:18.692890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe3aff-b6f4-4e18-a395-92cc711e4645 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:18.710382 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19a01e-4fd4-4526-80ab-58f0858a225e tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:18.719468 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1390/2785] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:17:18 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:17:18.720585 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe3aff-b6f4-4e18-a395-92cc711e4645 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:18.721192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19a01e-4fd4-4526-80ab-58f0858a225e tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:18.726494 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266c618033ec4b1d8a541a1ffc6a495d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08bd36b51ea940bab7f3c9be7244eb82', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:18.727385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.727676 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.728022 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] PATH_INFO: `/v3/projects/6eadcd11b47143289c14423da8bef3d4` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.729413 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6eadcd11b47143289c14423da8bef3d4)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:18.731677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:18.740023 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe3aff-b6f4-4e18-a395-92cc711e4645 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:18.745080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc1b48-7257-4194-91fa-3005e267b451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:18.745857 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc1b48-7257-4194-91fa-3005e267b451 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.746125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc1b48-7257-4194-91fa-3005e267b451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.746398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc1b48-7257-4194-91fa-3005e267b451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.747124 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc1b48-7257-4194-91fa-3005e267b451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:18.748718 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc1b48-7257-4194-91fa-3005e267b451 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:18.759282 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1391/2786] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:18.763305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc1b48-7257-4194-91fa-3005e267b451 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:18.775441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc1b48-7257-4194-91fa-3005e267b451 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:18.781970 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eadcd11b47143289c14423da8bef3d4', 'request_id': 'req-80b18e5f-3c35-426b-92e2-1b9465e4c65a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:18.787925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eadcd11b47143289c14423da8bef3d4', 'request_id': 'req-80b18e5f-3c35-426b-92e2-1b9465e4c65a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:18.794689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eadcd11b47143289c14423da8bef3d4', 'request_id': 'req-80b18e5f-3c35-426b-92e2-1b9465e4c65a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:18.809231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] The token cache is being invalidate because project 6eadcd11b47143289c14423da8bef3d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:17:18.809672 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:18.810308 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:18.812189 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1396/2787] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:17:18 2022] DELETE /identity/v3/projects/6eadcd11b47143289c14423da8bef3d4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:18.837993 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a13174-fc8f-4629-9fb5-348267a9fc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:18.838799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a13174-fc8f-4629-9fb5-348267a9fc3e admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.839148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a13174-fc8f-4629-9fb5-348267a9fc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.839417 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a13174-fc8f-4629-9fb5-348267a9fc3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.840301 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a13174-fc8f-4629-9fb5-348267a9fc3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:18.842025 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a13174-fc8f-4629-9fb5-348267a9fc3e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:18.852578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1392/2788] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:17:18.856877 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:18.858328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:18.872529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b18e5f-3c35-426b-92e2-1b9465e4c65a tempest-VolumeQuotaClassesTest-1354070693 tempest-VolumeQuotaClassesTest-1354070693-project-admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:18.884739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940dc8b2-5593-4c5d-85f6-bbf2fd79acfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:18.885848 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940dc8b2-5593-4c5d-85f6-bbf2fd79acfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.886139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940dc8b2-5593-4c5d-85f6-bbf2fd79acfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.886420 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940dc8b2-5593-4c5d-85f6-bbf2fd79acfb admin admin] PATH_INFO: `/v3/projects/09b54b24115e4074a79f93c255d4c0ca/users/2264f97424ae45ffa5770a66e95897c7/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.905442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940dc8b2-5593-4c5d-85f6-bbf2fd79acfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b54b24115e4074a79f93c255d4c0ca, user_id=2264f97424ae45ffa5770a66e95897c7, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:18.908855 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940dc8b2-5593-4c5d-85f6-bbf2fd79acfb admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:18.931332 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1397/2789] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:18 2022] PUT /identity/v3/projects/09b54b24115e4074a79f93c255d4c0ca/users/2264f97424ae45ffa5770a66e95897c7/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:18.937031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe996c-ddc2-42e1-bfc1-302c03c704bb None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:18.937332 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe996c-ddc2-42e1-bfc1-302c03c704bb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:18.937664 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe996c-ddc2-42e1-bfc1-302c03c704bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:18.966031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-fcbe996c-ddc2-42e1-bfc1-302c03c704bb None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:18.967856 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fcbe996c-ddc2-42e1-bfc1-302c03c704bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:19.050797 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940dc8b2-5593-4c5d-85f6-bbf2fd79acfb admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:19.052880 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-940dc8b2-5593-4c5d-85f6-bbf2fd79acfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:19.064534 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1393/2790] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:19.078144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe996c-ddc2-42e1-bfc1-302c03c704bb None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:19.079599 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fcbe996c-ddc2-42e1-bfc1-302c03c704bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:19.139333 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:19.144401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940dc8b2-5593-4c5d-85f6-bbf2fd79acfb admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:19.149945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:19.151075 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:19.151435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:19.151829 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] PATH_INFO: `/v3/users/bf2f5101f6554643bdcaa21b194377ec` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:19.152732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe996c-ddc2-42e1-bfc1-302c03c704bb None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:19.153059 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf2f5101f6554643bdcaa21b194377ec)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:19.155051 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:19.156763 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c5c57-f390-4f37-bfca-17d1086d76b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:19.157426 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c5c57-f390-4f37-bfca-17d1086d76b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:19.157743 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c5c57-f390-4f37-bfca-17d1086d76b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:19.158026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c5c57-f390-4f37-bfca-17d1086d76b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:19.168210 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c5c57-f390-4f37-bfca-17d1086d76b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:19.170141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c5c57-f390-4f37-bfca-17d1086d76b4 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:19.181755 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1394/2791] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:17:19.216262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf2f5101f6554643bdcaa21b194377ec'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:19.217581 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf2f5101f6554643bdcaa21b194377ec'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:19.227577 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf2f5101f6554643bdcaa21b194377ec'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:19.233063 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:19.245807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf2f5101f6554643bdcaa21b194377ec'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:19.246573 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf2f5101f6554643bdcaa21b194377ec'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:19.268724 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1398/2792] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:17:19 2022] DELETE /identity/v3/users/bf2f5101f6554643bdcaa21b194377ec => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:19.274647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c6546-a122-4967-bc0a-5872f1e7aff1 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:19.275015 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c6546-a122-4967-bc0a-5872f1e7aff1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:19.275385 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c6546-a122-4967-bc0a-5872f1e7aff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:19.297337 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e67c6546-a122-4967-bc0a-5872f1e7aff1 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:19.298780 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e67c6546-a122-4967-bc0a-5872f1e7aff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:19.379792 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1395/2793] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:19.389483 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:19.391042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:19.456325 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d873696-bd02-4e2d-b3d5-329d76f4bffb admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:19.459982 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f662f-24f3-4b5a-946b-ea0884c1da3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:19.460992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f662f-24f3-4b5a-946b-ea0884c1da3b service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:19.461276 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f662f-24f3-4b5a-946b-ea0884c1da3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:19.461549 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f662f-24f3-4b5a-946b-ea0884c1da3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:19.471438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f662f-24f3-4b5a-946b-ea0884c1da3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:19.472956 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f662f-24f3-4b5a-946b-ea0884c1da3b service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:19.483097 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1399/2794] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:17:19.960970 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c6546-a122-4967-bc0a-5872f1e7aff1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:19.962818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e67c6546-a122-4967-bc0a-5872f1e7aff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:19.975965 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c6546-a122-4967-bc0a-5872f1e7aff1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:19.983484 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:19.984464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:19.984799 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:19.985067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] PATH_INFO: `/v3/projects/65654bd6e75542a093b478139cadf49b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:19.986607 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65654bd6e75542a093b478139cadf49b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:19.988656 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:20.078343 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65654bd6e75542a093b478139cadf49b', 'request_id': 'req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.085453 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65654bd6e75542a093b478139cadf49b', 'request_id': 'req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.094048 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65654bd6e75542a093b478139cadf49b', 'request_id': 'req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.109974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] The token cache is being invalidate because project 65654bd6e75542a093b478139cadf49b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:17:20.110349 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.114178 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.116058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1396/2795] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:17:19 2022] DELETE /identity/v3/projects/65654bd6e75542a093b478139cadf49b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:20.120080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f662f-24f3-4b5a-946b-ea0884c1da3b service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:20.145826 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f662f-24f3-4b5a-946b-ea0884c1da3b service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:20.214903 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604baaa-2973-4aaa-a0e7-c13feec0dae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:20.215845 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604baaa-2973-4aaa-a0e7-c13feec0dae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:20.216131 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604baaa-2973-4aaa-a0e7-c13feec0dae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:20.217333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604baaa-2973-4aaa-a0e7-c13feec0dae1 admin admin] PATH_INFO: `/v3/users/266c618033ec4b1d8a541a1ffc6a495d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:20.218712 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604baaa-2973-4aaa-a0e7-c13feec0dae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=266c618033ec4b1d8a541a1ffc6a495d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:20.222202 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604baaa-2973-4aaa-a0e7-c13feec0dae1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:20.300657 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8604baaa-2973-4aaa-a0e7-c13feec0dae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '266c618033ec4b1d8a541a1ffc6a495d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.302576 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8604baaa-2973-4aaa-a0e7-c13feec0dae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '266c618033ec4b1d8a541a1ffc6a495d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.315940 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8604baaa-2973-4aaa-a0e7-c13feec0dae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '266c618033ec4b1d8a541a1ffc6a495d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.332500 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8604baaa-2973-4aaa-a0e7-c13feec0dae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '266c618033ec4b1d8a541a1ffc6a495d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.333277 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8604baaa-2973-4aaa-a0e7-c13feec0dae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '266c618033ec4b1d8a541a1ffc6a495d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.351920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:20.358805 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1400/2796] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:17:20 2022] DELETE /identity/v3/users/266c618033ec4b1d8a541a1ffc6a495d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:20.681266 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:20.682751 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:20.705205 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43eb9f-b28c-4c8f-bf9b-5175564034b6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:20.813073 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:20.814860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:20.815428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:20.815805 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a admin admin] PATH_INFO: `/v3/projects/08bd36b51ea940bab7f3c9be7244eb82` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:20.817308 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a admin admin] RBAC: Authorizing `identity:delete_project(project_id=08bd36b51ea940bab7f3c9be7244eb82)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:20.819618 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:20.843922 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:20.894139 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08bd36b51ea940bab7f3c9be7244eb82', 'request_id': 'req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.901167 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08bd36b51ea940bab7f3c9be7244eb82', 'request_id': 'req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.907566 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08bd36b51ea940bab7f3c9be7244eb82', 'request_id': 'req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.931347 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a admin admin] The token cache is being invalidate because project 08bd36b51ea940bab7f3c9be7244eb82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:17:20.931863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.932806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1c496a09-5d2a-4c8c-acbd-5adbc9b1087a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:20.935905 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1397/2797] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:17:20 2022] DELETE /identity/v3/projects/08bd36b51ea940bab7f3c9be7244eb82 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:20.943370 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579f1a4-94d8-4c8e-b8ec-5c39ccbf35bc None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:20.943862 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579f1a4-94d8-4c8e-b8ec-5c39ccbf35bc None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:20.943947 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579f1a4-94d8-4c8e-b8ec-5c39ccbf35bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:20.996783 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7579f1a4-94d8-4c8e-b8ec-5c39ccbf35bc None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:20.998052 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7579f1a4-94d8-4c8e-b8ec-5c39ccbf35bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:21.085047 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1401/2798] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:21.097109 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbc2cb-af3c-4e2c-ad6d-1e9a6ee597cc None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:21.097571 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbc2cb-af3c-4e2c-ad6d-1e9a6ee597cc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:21.098071 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbc2cb-af3c-4e2c-ad6d-1e9a6ee597cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:21.135464 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-86fbc2cb-af3c-4e2c-ad6d-1e9a6ee597cc None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:21.136852 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-86fbc2cb-af3c-4e2c-ad6d-1e9a6ee597cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:21.146348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1398/2799] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:21.150765 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579f1a4-94d8-4c8e-b8ec-5c39ccbf35bc None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:21.161139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579f1a4-94d8-4c8e-b8ec-5c39ccbf35bc None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:21.173678 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e25a98-122e-46b8-8698-4291930468ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:21.174412 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e25a98-122e-46b8-8698-4291930468ae admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:21.174745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e25a98-122e-46b8-8698-4291930468ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:21.174976 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e25a98-122e-46b8-8698-4291930468ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:21.175721 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e25a98-122e-46b8-8698-4291930468ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:17:21.176028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e25a98-122e-46b8-8698-4291930468ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:21.177688 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e25a98-122e-46b8-8698-4291930468ae admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:21.190932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1402/2800] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:17:21.195389 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fbc2cb-af3c-4e2c-ad6d-1e9a6ee597cc None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:21.204692 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fbc2cb-af3c-4e2c-ad6d-1e9a6ee597cc None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:21.209787 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7ae74-1bf0-4082-9b49-061dbd0a1c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:21.211163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7ae74-1bf0-4082-9b49-061dbd0a1c4d admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:21.211419 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7ae74-1bf0-4082-9b49-061dbd0a1c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:21.211700 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7ae74-1bf0-4082-9b49-061dbd0a1c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:21.212508 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7ae74-1bf0-4082-9b49-061dbd0a1c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:21.214822 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7ae74-1bf0-4082-9b49-061dbd0a1c4d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:21.254631 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1399/2801] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:17:21 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:21.259050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e25a98-122e-46b8-8698-4291930468ae admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:21.269216 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e25a98-122e-46b8-8698-4291930468ae admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:21.331490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e673030-8b08-495a-babd-43e383b7279c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:21.332276 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e673030-8b08-495a-babd-43e383b7279c admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:21.332660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e673030-8b08-495a-babd-43e383b7279c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:21.333005 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e673030-8b08-495a-babd-43e383b7279c admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:21.333808 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e673030-8b08-495a-babd-43e383b7279c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:21.336227 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e673030-8b08-495a-babd-43e383b7279c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:21.362385 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1403/2802] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:17:21 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:21.364030 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7ae74-1bf0-4082-9b49-061dbd0a1c4d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:21.375222 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7ae74-1bf0-4082-9b49-061dbd0a1c4d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:21.382738 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30f526-2f83-44db-8bf6-40ec4cc1fb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:21.384638 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30f526-2f83-44db-8bf6-40ec4cc1fb41 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:21.384923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30f526-2f83-44db-8bf6-40ec4cc1fb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:21.385199 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30f526-2f83-44db-8bf6-40ec4cc1fb41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:21.386017 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30f526-2f83-44db-8bf6-40ec4cc1fb41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:21.387713 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30f526-2f83-44db-8bf6-40ec4cc1fb41 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:21.399141 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1400/2803] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:21.403798 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e673030-8b08-495a-babd-43e383b7279c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:21.414980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e673030-8b08-495a-babd-43e383b7279c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:21.419567 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f694c-49c4-4e1f-be4d-8a700cda0523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:21.420245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f694c-49c4-4e1f-be4d-8a700cda0523 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:21.420517 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f694c-49c4-4e1f-be4d-8a700cda0523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:21.420795 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f694c-49c4-4e1f-be4d-8a700cda0523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:21.421518 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f694c-49c4-4e1f-be4d-8a700cda0523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:21.423223 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f694c-49c4-4e1f-be4d-8a700cda0523 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:21.434074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1404/2804] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:21.439057 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30f526-2f83-44db-8bf6-40ec4cc1fb41 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:21.451299 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30f526-2f83-44db-8bf6-40ec4cc1fb41 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:21.456346 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd9109-0ab9-4791-b278-528aa7c35b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:21.457312 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd9109-0ab9-4791-b278-528aa7c35b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:21.457685 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd9109-0ab9-4791-b278-528aa7c35b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:21.458023 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd9109-0ab9-4791-b278-528aa7c35b44 admin admin] PATH_INFO: `/v3/projects/0cf97b3228fb4369a21d800fbd734d36/users/e98da577be0143a38b96342792ea9376/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:21.476732 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd9109-0ab9-4791-b278-528aa7c35b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf97b3228fb4369a21d800fbd734d36, user_id=e98da577be0143a38b96342792ea9376, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:21.479459 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd9109-0ab9-4791-b278-528aa7c35b44 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:21.497696 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1401/2805] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:21 2022] PUT /identity/v3/projects/0cf97b3228fb4369a21d800fbd734d36/users/e98da577be0143a38b96342792ea9376/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:21.515434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad475c-e3ca-4791-b3ae-77adc362fe22 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:21.515874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad475c-e3ca-4791-b3ae-77adc362fe22 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:21.516158 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad475c-e3ca-4791-b3ae-77adc362fe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:21.546905 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-42ad475c-e3ca-4791-b3ae-77adc362fe22 None None] MFA Rules not processed for user `e98da577be0143a38b96342792ea9376`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:21.548143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-42ad475c-e3ca-4791-b3ae-77adc362fe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:21.619718 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:21.630157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1405/2806] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:21.634874 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd9109-0ab9-4791-b278-528aa7c35b44 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:21.708553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd9109-0ab9-4791-b278-528aa7c35b44 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:21.718131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6ba44-7c9c-4f0a-94df-f317df8f5840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:21.718825 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6ba44-7c9c-4f0a-94df-f317df8f5840 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:21.719103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6ba44-7c9c-4f0a-94df-f317df8f5840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:21.719377 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6ba44-7c9c-4f0a-94df-f317df8f5840 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:21.720082 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6ba44-7c9c-4f0a-94df-f317df8f5840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:21.722115 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6ba44-7c9c-4f0a-94df-f317df8f5840 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:21.759056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1402/2807] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:17:21 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:17:21.763249 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad475c-e3ca-4791-b3ae-77adc362fe22 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:21.775713 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad475c-e3ca-4791-b3ae-77adc362fe22 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:21.869284 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3e352-7620-4e7d-ac0f-a8a7b4f995aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:21.870270 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3e352-7620-4e7d-ac0f-a8a7b4f995aa admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:21.870579 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3e352-7620-4e7d-ac0f-a8a7b4f995aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:21.870923 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3e352-7620-4e7d-ac0f-a8a7b4f995aa admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:21.871675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3e352-7620-4e7d-ac0f-a8a7b4f995aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:21.873811 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3e352-7620-4e7d-ac0f-a8a7b4f995aa admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:21.901710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1406/2808] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:17:21 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:21.903077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6ba44-7c9c-4f0a-94df-f317df8f5840 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:21.916112 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6ba44-7c9c-4f0a-94df-f317df8f5840 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:21.930936 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfb555-8d9e-46d7-8362-a299fc476aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:21.931830 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfb555-8d9e-46d7-8362-a299fc476aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:21.932208 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfb555-8d9e-46d7-8362-a299fc476aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:21.932614 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfb555-8d9e-46d7-8362-a299fc476aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:21.933454 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfb555-8d9e-46d7-8362-a299fc476aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:21.936409 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfb555-8d9e-46d7-8362-a299fc476aa5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:21.961545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3e352-7620-4e7d-ac0f-a8a7b4f995aa admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:21.971107 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1403/2809] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 10:17:21.977816 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3e352-7620-4e7d-ac0f-a8a7b4f995aa admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:21.990245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf35a5-b4e2-41d0-8f74-5021406a23da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:21.992513 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf35a5-b4e2-41d0-8f74-5021406a23da admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:21.994139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf35a5-b4e2-41d0-8f74-5021406a23da admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:21.994437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf35a5-b4e2-41d0-8f74-5021406a23da admin admin] PATH_INFO: `/v3/projects/4d329250ba734828bc3ff6dc9e8eae49/users/bbae3e8be86d4acbb60eb7bb3f9d14e4/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:22.019093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf35a5-b4e2-41d0-8f74-5021406a23da admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d329250ba734828bc3ff6dc9e8eae49, user_id=bbae3e8be86d4acbb60eb7bb3f9d14e4, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:22.022235 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf35a5-b4e2-41d0-8f74-5021406a23da admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:22.059368 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1407/2810] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:21 2022] PUT /identity/v3/projects/4d329250ba734828bc3ff6dc9e8eae49/users/bbae3e8be86d4acbb60eb7bb3f9d14e4/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:22.063677 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfb555-8d9e-46d7-8362-a299fc476aa5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:22.079884 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfb555-8d9e-46d7-8362-a299fc476aa5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:22.087195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:22.170126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0b968-f3b7-4156-8624-ffe7bdd318a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:22.171178 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0b968-f3b7-4156-8624-ffe7bdd318a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:22.171546 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0b968-f3b7-4156-8624-ffe7bdd318a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:22.171893 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0b968-f3b7-4156-8624-ffe7bdd318a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:22.172757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0b968-f3b7-4156-8624-ffe7bdd318a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:22.176694 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0b968-f3b7-4156-8624-ffe7bdd318a9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:22.203725 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1404/2811] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:22.208890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf35a5-b4e2-41d0-8f74-5021406a23da admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:22.226547 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf35a5-b4e2-41d0-8f74-5021406a23da admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:22.233948 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194628aa-3895-44a5-ae61-a8f3afea612e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:22.234958 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194628aa-3895-44a5-ae61-a8f3afea612e admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:22.235311 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194628aa-3895-44a5-ae61-a8f3afea612e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:22.235660 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194628aa-3895-44a5-ae61-a8f3afea612e admin admin] PATH_INFO: `/v3/projects/4d329250ba734828bc3ff6dc9e8eae49/users/bbae3e8be86d4acbb60eb7bb3f9d14e4/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:22.238854 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194628aa-3895-44a5-ae61-a8f3afea612e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d329250ba734828bc3ff6dc9e8eae49, user_id=bbae3e8be86d4acbb60eb7bb3f9d14e4, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:22.242840 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194628aa-3895-44a5-ae61-a8f3afea612e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:22.261529 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1408/2812] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:22 2022] PUT /identity/v3/projects/4d329250ba734828bc3ff6dc9e8eae49/users/bbae3e8be86d4acbb60eb7bb3f9d14e4/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:22.275597 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48b0eb-a61d-4068-abee-c7a865a0f4ed None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:22.275965 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48b0eb-a61d-4068-abee-c7a865a0f4ed None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:22.276195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48b0eb-a61d-4068-abee-c7a865a0f4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:22.340191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9a48b0eb-a61d-4068-abee-c7a865a0f4ed None None] MFA Rules not processed for user `bbae3e8be86d4acbb60eb7bb3f9d14e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:22.341852 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9a48b0eb-a61d-4068-abee-c7a865a0f4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:22.449399 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1405/2813] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:22.466140 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194628aa-3895-44a5-ae61-a8f3afea612e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:22.469885 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-194628aa-3895-44a5-ae61-a8f3afea612e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:22.561561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194628aa-3895-44a5-ae61-a8f3afea612e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:22.570986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c172e-c54d-410d-8d73-cecbcdd37fe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:22.570986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c172e-c54d-410d-8d73-cecbcdd37fe2 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:22.570986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c172e-c54d-410d-8d73-cecbcdd37fe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:22.570986 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c172e-c54d-410d-8d73-cecbcdd37fe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:22.582831 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c172e-c54d-410d-8d73-cecbcdd37fe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:22.585442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c172e-c54d-410d-8d73-cecbcdd37fe2 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:22.597142 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1409/2814] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:17:22.744270 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48b0eb-a61d-4068-abee-c7a865a0f4ed None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:22.761375 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48b0eb-a61d-4068-abee-c7a865a0f4ed None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:22.765026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90ebf7-55d9-417e-98b7-91b62a478e8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:22.765723 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90ebf7-55d9-417e-98b7-91b62a478e8f service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:22.766012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90ebf7-55d9-417e-98b7-91b62a478e8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:22.766293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90ebf7-55d9-417e-98b7-91b62a478e8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:22.849652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90ebf7-55d9-417e-98b7-91b62a478e8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:22.851131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90ebf7-55d9-417e-98b7-91b62a478e8f service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:22.868180 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1406/2815] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:17:24.085373 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317275e-ee8e-4bd4-ac2c-156a837d18a1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:24.086269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317275e-ee8e-4bd4-ac2c-156a837d18a1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:24.086456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317275e-ee8e-4bd4-ac2c-156a837d18a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:24.171481 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b317275e-ee8e-4bd4-ac2c-156a837d18a1 None None] MFA Rules not processed for user `2264f97424ae45ffa5770a66e95897c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:24.173063 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b317275e-ee8e-4bd4-ac2c-156a837d18a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:24.249104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:24.292896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1410/2816] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:24.297611 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90ebf7-55d9-417e-98b7-91b62a478e8f service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:24.299363 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-cb90ebf7-55d9-417e-98b7-91b62a478e8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:24.355475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:24.395163 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90ebf7-55d9-417e-98b7-91b62a478e8f service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:24.400441 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b8974-ff02-4bd2-898e-c414fcbb1f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:24.401165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b8974-ff02-4bd2-898e-c414fcbb1f09 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:24.401451 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b8974-ff02-4bd2-898e-c414fcbb1f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:24.401965 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b8974-ff02-4bd2-898e-c414fcbb1f09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:24.402807 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b8974-ff02-4bd2-898e-c414fcbb1f09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:24.405031 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b8974-ff02-4bd2-898e-c414fcbb1f09 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:24.448722 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1407/2817] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:17:24 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:17:24.449558 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317275e-ee8e-4bd4-ac2c-156a837d18a1 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:24.462684 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317275e-ee8e-4bd4-ac2c-156a837d18a1 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:24.539359 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9730390-8ec4-436f-a7ce-b241a23761ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:24.539968 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9730390-8ec4-436f-a7ce-b241a23761ed admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:24.540248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9730390-8ec4-436f-a7ce-b241a23761ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:24.541927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9730390-8ec4-436f-a7ce-b241a23761ed admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:24.541927 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9730390-8ec4-436f-a7ce-b241a23761ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:24.544804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9730390-8ec4-436f-a7ce-b241a23761ed admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:24.570745 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1411/2818] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:17:24 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:17:24.575184 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b8974-ff02-4bd2-898e-c414fcbb1f09 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:24.590918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b8974-ff02-4bd2-898e-c414fcbb1f09 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:24.595403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9b3f9-afbf-4556-b002-e452594402d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:24.596350 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9b3f9-afbf-4556-b002-e452594402d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:24.596579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9b3f9-afbf-4556-b002-e452594402d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:24.596997 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9b3f9-afbf-4556-b002-e452594402d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:24.597690 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9b3f9-afbf-4556-b002-e452594402d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:24.599674 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9b3f9-afbf-4556-b002-e452594402d7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:24.612665 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1408/2819] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:24.617932 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9730390-8ec4-436f-a7ce-b241a23761ed admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:24.634293 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9730390-8ec4-436f-a7ce-b241a23761ed admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:24.647658 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad22700-faa3-4090-ab1b-fa4f09158342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:24.647658 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad22700-faa3-4090-ab1b-fa4f09158342 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:24.647658 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad22700-faa3-4090-ab1b-fa4f09158342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:24.647658 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad22700-faa3-4090-ab1b-fa4f09158342 admin admin] PATH_INFO: `/v3/projects/8a785e9fae27434bbd0cac2396834fe8/users/ace3c4ac54514056b9a48abf9a3cb64a/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:24.672457 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad22700-faa3-4090-ab1b-fa4f09158342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a785e9fae27434bbd0cac2396834fe8, user_id=ace3c4ac54514056b9a48abf9a3cb64a, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:24.676241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad22700-faa3-4090-ab1b-fa4f09158342 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:24.693830 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1412/2820] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:24 2022] PUT /identity/v3/projects/8a785e9fae27434bbd0cac2396834fe8/users/ace3c4ac54514056b9a48abf9a3cb64a/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:24.698173 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9b3f9-afbf-4556-b002-e452594402d7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:24.709245 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9b3f9-afbf-4556-b002-e452594402d7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:24.780171 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70906ac9-bb0c-4ba3-a6d7-fc45a1cba9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:24.780772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70906ac9-bb0c-4ba3-a6d7-fc45a1cba9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:24.781062 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70906ac9-bb0c-4ba3-a6d7-fc45a1cba9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:24.781345 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70906ac9-bb0c-4ba3-a6d7-fc45a1cba9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:24.782145 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70906ac9-bb0c-4ba3-a6d7-fc45a1cba9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:24.785238 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70906ac9-bb0c-4ba3-a6d7-fc45a1cba9b4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:24.798295 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1409/2821] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:24.802606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad22700-faa3-4090-ab1b-fa4f09158342 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:24.814467 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad22700-faa3-4090-ab1b-fa4f09158342 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:24.819632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75bc7f-b04c-4cbf-a9aa-2f9870022a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:24.821665 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75bc7f-b04c-4cbf-a9aa-2f9870022a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:24.822143 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75bc7f-b04c-4cbf-a9aa-2f9870022a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:24.822432 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75bc7f-b04c-4cbf-a9aa-2f9870022a2f admin admin] PATH_INFO: `/v3/projects/8a785e9fae27434bbd0cac2396834fe8/users/ace3c4ac54514056b9a48abf9a3cb64a/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:24.825461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75bc7f-b04c-4cbf-a9aa-2f9870022a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a785e9fae27434bbd0cac2396834fe8, user_id=ace3c4ac54514056b9a48abf9a3cb64a, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:24.828253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75bc7f-b04c-4cbf-a9aa-2f9870022a2f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:24.840789 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1413/2822] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:24 2022] PUT /identity/v3/projects/8a785e9fae27434bbd0cac2396834fe8/users/ace3c4ac54514056b9a48abf9a3cb64a/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:25.674058 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70906ac9-bb0c-4ba3-a6d7-fc45a1cba9b4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:25.687990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70906ac9-bb0c-4ba3-a6d7-fc45a1cba9b4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:25.766860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:25.768898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:25.769191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:25.769472 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] PATH_INFO: `/v3/users/e98da577be0143a38b96342792ea9376` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:25.770852 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] RBAC: Authorizing `identity:delete_user(user_id=e98da577be0143a38b96342792ea9376)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:25.772818 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:25.827309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:25.836267 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e98da577be0143a38b96342792ea9376'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:25.837095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e98da577be0143a38b96342792ea9376'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:25.848702 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e98da577be0143a38b96342792ea9376'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:25.865518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e98da577be0143a38b96342792ea9376'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:25.866396 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e98da577be0143a38b96342792ea9376'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:25.887360 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1410/2823] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:17:25 2022] DELETE /identity/v3/users/e98da577be0143a38b96342792ea9376 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:17:25.889426 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474faed6-125a-4f44-8662-5ac2e9770c31 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:25.890560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474faed6-125a-4f44-8662-5ac2e9770c31 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:25.890560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474faed6-125a-4f44-8662-5ac2e9770c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:25.912371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-474faed6-125a-4f44-8662-5ac2e9770c31 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:25.913605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-474faed6-125a-4f44-8662-5ac2e9770c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:25.995814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1414/2824] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 10:17:26.009099 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:26.010020 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:26.076305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3fd63-df8e-41b8-b98f-5d6d70d1acad admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:26.080258 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ee1f3-76d5-497b-acb1-d41cf86d953a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:26.081098 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ee1f3-76d5-497b-acb1-d41cf86d953a service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:26.081489 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ee1f3-76d5-497b-acb1-d41cf86d953a service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:26.081898 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ee1f3-76d5-497b-acb1-d41cf86d953a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:26.092790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ee1f3-76d5-497b-acb1-d41cf86d953a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:26.094731 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ee1f3-76d5-497b-acb1-d41cf86d953a service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:26.105990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1411/2825] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:17:26.434104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474faed6-125a-4f44-8662-5ac2e9770c31 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:26.435590 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-474faed6-125a-4f44-8662-5ac2e9770c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:26.459518 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474faed6-125a-4f44-8662-5ac2e9770c31 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:26.474490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:26.475890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:26.476708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:26.477100 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] PATH_INFO: `/v3/projects/0cf97b3228fb4369a21d800fbd734d36` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:26.478532 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf97b3228fb4369a21d800fbd734d36)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:26.480645 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:26.521842 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:26.543471 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf97b3228fb4369a21d800fbd734d36', 'request_id': 'req-45c99810-41bb-4fa3-a168-ccd798ab4175'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:26.554793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf97b3228fb4369a21d800fbd734d36', 'request_id': 'req-45c99810-41bb-4fa3-a168-ccd798ab4175'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:26.562737 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf97b3228fb4369a21d800fbd734d36', 'request_id': 'req-45c99810-41bb-4fa3-a168-ccd798ab4175'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:26.584758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] The token cache is being invalidate because project 0cf97b3228fb4369a21d800fbd734d36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:17:26.585899 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:26.586650 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:26.588540 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1415/2826] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:17:26 2022] DELETE /identity/v3/projects/0cf97b3228fb4369a21d800fbd734d36 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:17:26.594817 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ee1f3-76d5-497b-acb1-d41cf86d953a service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:26.607551 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ee1f3-76d5-497b-acb1-d41cf86d953a service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:26.674533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8821d-4e72-4172-ab6e-9b47c85cb187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:26.675385 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8821d-4e72-4172-ab6e-9b47c85cb187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:26.675906 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8821d-4e72-4172-ab6e-9b47c85cb187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:26.676273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8821d-4e72-4172-ab6e-9b47c85cb187 admin admin] PATH_INFO: `/v3/users/bbae3e8be86d4acbb60eb7bb3f9d14e4` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:26.677872 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8821d-4e72-4172-ab6e-9b47c85cb187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbae3e8be86d4acbb60eb7bb3f9d14e4)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:26.680918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8821d-4e72-4172-ab6e-9b47c85cb187 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:26.741237 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-71e8821d-4e72-4172-ab6e-9b47c85cb187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbae3e8be86d4acbb60eb7bb3f9d14e4'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:26.741995 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-71e8821d-4e72-4172-ab6e-9b47c85cb187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbae3e8be86d4acbb60eb7bb3f9d14e4'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:26.752993 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-71e8821d-4e72-4172-ab6e-9b47c85cb187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbae3e8be86d4acbb60eb7bb3f9d14e4'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:26.761556 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:26.768276 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-71e8821d-4e72-4172-ab6e-9b47c85cb187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbae3e8be86d4acbb60eb7bb3f9d14e4'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:26.769170 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-71e8821d-4e72-4172-ab6e-9b47c85cb187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbae3e8be86d4acbb60eb7bb3f9d14e4'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:26.787772 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1412/2827] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:17:26 2022] DELETE /identity/v3/users/bbae3e8be86d4acbb60eb7bb3f9d14e4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 10:17:27.184119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:27.186382 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:27.211272 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c99810-41bb-4fa3-a168-ccd798ab4175 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:27.377675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0b3e4-d026-4009-8cc3-a9f375b0808f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:27.378892 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0b3e4-d026-4009-8cc3-a9f375b0808f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:27.379323 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0b3e4-d026-4009-8cc3-a9f375b0808f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:27.379764 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0b3e4-d026-4009-8cc3-a9f375b0808f admin admin] PATH_INFO: `/v3/projects/4d329250ba734828bc3ff6dc9e8eae49` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:27.381267 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0b3e4-d026-4009-8cc3-a9f375b0808f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d329250ba734828bc3ff6dc9e8eae49)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:27.383456 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0b3e4-d026-4009-8cc3-a9f375b0808f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:27.504997 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-11a0b3e4-d026-4009-8cc3-a9f375b0808f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d329250ba734828bc3ff6dc9e8eae49', 'request_id': 'req-11a0b3e4-d026-4009-8cc3-a9f375b0808f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:27.536743 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-11a0b3e4-d026-4009-8cc3-a9f375b0808f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d329250ba734828bc3ff6dc9e8eae49', 'request_id': 'req-11a0b3e4-d026-4009-8cc3-a9f375b0808f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:27.555435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-11a0b3e4-d026-4009-8cc3-a9f375b0808f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d329250ba734828bc3ff6dc9e8eae49', 'request_id': 'req-11a0b3e4-d026-4009-8cc3-a9f375b0808f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:27.600087 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-11a0b3e4-d026-4009-8cc3-a9f375b0808f admin admin] The token cache is being invalidate because project 4d329250ba734828bc3ff6dc9e8eae49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:17:27.600710 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-11a0b3e4-d026-4009-8cc3-a9f375b0808f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:27.601824 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-11a0b3e4-d026-4009-8cc3-a9f375b0808f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:27.607038 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1416/2828] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:17:27 2022] DELETE /identity/v3/projects/4d329250ba734828bc3ff6dc9e8eae49 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:27.611676 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c396fe-3930-477c-a10d-0e6e0fdf216a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:27.612093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c396fe-3930-477c-a10d-0e6e0fdf216a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:27.612468 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c396fe-3930-477c-a10d-0e6e0fdf216a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:27.644517 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e3c396fe-3930-477c-a10d-0e6e0fdf216a None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:27.646012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e3c396fe-3930-477c-a10d-0e6e0fdf216a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:27.742947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1413/2829] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:27.750988 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8104033-1f65-4874-b0d8-58cc69cd7e81 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:27.754250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8104033-1f65-4874-b0d8-58cc69cd7e81 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:27.754908 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8104033-1f65-4874-b0d8-58cc69cd7e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:27.802920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e8104033-1f65-4874-b0d8-58cc69cd7e81 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:27.804437 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e8104033-1f65-4874-b0d8-58cc69cd7e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:27.824991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1417/2830] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:27.829357 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c396fe-3930-477c-a10d-0e6e0fdf216a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:27.842252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c396fe-3930-477c-a10d-0e6e0fdf216a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:27.847501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3e2dd-70e1-4f84-b39e-f5b2671728af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:27.848555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3e2dd-70e1-4f84-b39e-f5b2671728af admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:27.848847 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3e2dd-70e1-4f84-b39e-f5b2671728af admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:27.849126 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3e2dd-70e1-4f84-b39e-f5b2671728af admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:27.849956 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3e2dd-70e1-4f84-b39e-f5b2671728af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 10:17:27.850305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3e2dd-70e1-4f84-b39e-f5b2671728af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:27.851926 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3e2dd-70e1-4f84-b39e-f5b2671728af admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:27.889100 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1414/2831] 10.222.0.79 () {58 vars in 1152 bytes} [Mon Jul 4 10:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 10:17:27.893459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8104033-1f65-4874-b0d8-58cc69cd7e81 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:27.903878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8104033-1f65-4874-b0d8-58cc69cd7e81 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:27.909670 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd2b53-04a3-4995-82ed-398649d69337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:27.910732 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd2b53-04a3-4995-82ed-398649d69337 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:27.911306 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd2b53-04a3-4995-82ed-398649d69337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:27.911758 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd2b53-04a3-4995-82ed-398649d69337 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:27.912600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd2b53-04a3-4995-82ed-398649d69337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:27.914733 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd2b53-04a3-4995-82ed-398649d69337 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:27.997013 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1418/2832] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:17:27 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:17:28.002758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3e2dd-70e1-4f84-b39e-f5b2671728af admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:28.026274 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3e2dd-70e1-4f84-b39e-f5b2671728af admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:28.146298 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86073da0-74b8-45a8-bbf5-f97ace26429b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:28.147291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86073da0-74b8-45a8-bbf5-f97ace26429b admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:28.147616 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86073da0-74b8-45a8-bbf5-f97ace26429b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:28.147946 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86073da0-74b8-45a8-bbf5-f97ace26429b admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:28.148984 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86073da0-74b8-45a8-bbf5-f97ace26429b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:28.152096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86073da0-74b8-45a8-bbf5-f97ace26429b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:28.201775 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1415/2833] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:17:28 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:28.205716 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd2b53-04a3-4995-82ed-398649d69337 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:28.222229 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd2b53-04a3-4995-82ed-398649d69337 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:28.237751 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad44864-8899-4d2a-93fb-5ff2e42930b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:28.239479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad44864-8899-4d2a-93fb-5ff2e42930b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:28.239744 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad44864-8899-4d2a-93fb-5ff2e42930b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:28.240920 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad44864-8899-4d2a-93fb-5ff2e42930b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:28.245215 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad44864-8899-4d2a-93fb-5ff2e42930b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:28.251753 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad44864-8899-4d2a-93fb-5ff2e42930b0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:28.276018 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1419/2834] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:28.280281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86073da0-74b8-45a8-bbf5-f97ace26429b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:28.295319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86073da0-74b8-45a8-bbf5-f97ace26429b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:28.301450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13815e-5f11-4509-8b98-44d7e72ece73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:28.302370 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13815e-5f11-4509-8b98-44d7e72ece73 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:28.302757 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13815e-5f11-4509-8b98-44d7e72ece73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:28.303134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13815e-5f11-4509-8b98-44d7e72ece73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:28.304000 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13815e-5f11-4509-8b98-44d7e72ece73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:28.305880 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13815e-5f11-4509-8b98-44d7e72ece73 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:28.320179 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1416/2835] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:28.324157 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad44864-8899-4d2a-93fb-5ff2e42930b0 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:28.334623 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:28.352357 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad44864-8899-4d2a-93fb-5ff2e42930b0 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:28.362561 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3850-ddde-4334-ab17-c99b5a161e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:28.364340 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3850-ddde-4334-ab17-c99b5a161e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:28.364830 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3850-ddde-4334-ab17-c99b5a161e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:28.365266 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3850-ddde-4334-ab17-c99b5a161e2a admin admin] PATH_INFO: `/v3/projects/3d5813e628834a86b353d67cbc6d8cec/users/656ac2482c094050b48a529ee5e84bd0/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:28.385347 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f3850-ddde-4334-ab17-c99b5a161e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d5813e628834a86b353d67cbc6d8cec, user_id=656ac2482c094050b48a529ee5e84bd0, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:28.389156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f3850-ddde-4334-ab17-c99b5a161e2a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:28.428317 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1420/2836] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:28 2022] PUT /identity/v3/projects/3d5813e628834a86b353d67cbc6d8cec/users/656ac2482c094050b48a529ee5e84bd0/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:28.486601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1e4e3-5488-4680-9d90-95d85509ff00 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:28.487721 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1e4e3-5488-4680-9d90-95d85509ff00 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:28.488119 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1e4e3-5488-4680-9d90-95d85509ff00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:28.550281 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-77c1e4e3-5488-4680-9d90-95d85509ff00 None None] MFA Rules not processed for user `656ac2482c094050b48a529ee5e84bd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:28.551975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-77c1e4e3-5488-4680-9d90-95d85509ff00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:28.623499 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:28.689509 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3850-ddde-4334-ab17-c99b5a161e2a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:28.697004 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1417/2837] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:28.767196 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3850-ddde-4334-ab17-c99b5a161e2a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:28.774050 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929816d-37cb-486e-ab03-f1106c677c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:28.774446 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929816d-37cb-486e-ab03-f1106c677c07 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:28.774728 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929816d-37cb-486e-ab03-f1106c677c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:28.775009 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929816d-37cb-486e-ab03-f1106c677c07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:28.775772 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929816d-37cb-486e-ab03-f1106c677c07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:28.791937 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929816d-37cb-486e-ab03-f1106c677c07 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:28.836844 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1421/2838] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:17:28 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:28.836986 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1e4e3-5488-4680-9d90-95d85509ff00 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:28.856298 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1e4e3-5488-4680-9d90-95d85509ff00 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:29.035958 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0b350-6eba-401a-9045-bcd229e11066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:29.036662 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0b350-6eba-401a-9045-bcd229e11066 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:29.036947 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0b350-6eba-401a-9045-bcd229e11066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:29.037223 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0b350-6eba-401a-9045-bcd229e11066 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:29.037969 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0b350-6eba-401a-9045-bcd229e11066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:29.039975 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0b350-6eba-401a-9045-bcd229e11066 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:29.087554 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1418/2839] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:17:28 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 10:17:29.091614 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929816d-37cb-486e-ab03-f1106c677c07 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:29.107586 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929816d-37cb-486e-ab03-f1106c677c07 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:29.117910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f16c78-fb96-42c7-863d-2a039e631d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:29.118925 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f16c78-fb96-42c7-863d-2a039e631d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:29.129434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f16c78-fb96-42c7-863d-2a039e631d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:29.130176 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f16c78-fb96-42c7-863d-2a039e631d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:29.131149 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f16c78-fb96-42c7-863d-2a039e631d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:29.154061 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f16c78-fb96-42c7-863d-2a039e631d59 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:29.188610 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0b350-6eba-401a-9045-bcd229e11066 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:29.188999 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1422/2840] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:29.204689 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0b350-6eba-401a-9045-bcd229e11066 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:29.218279 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5586572-37f3-4ff7-9ced-9068c39aa9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:29.218983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5586572-37f3-4ff7-9ced-9068c39aa9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:29.219273 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5586572-37f3-4ff7-9ced-9068c39aa9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:29.219556 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5586572-37f3-4ff7-9ced-9068c39aa9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:29.220311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5586572-37f3-4ff7-9ced-9068c39aa9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:29.224432 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5586572-37f3-4ff7-9ced-9068c39aa9d8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:29.257232 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1419/2841] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:29.262512 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f16c78-fb96-42c7-863d-2a039e631d59 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:29.278874 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f16c78-fb96-42c7-863d-2a039e631d59 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:29.293702 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212d87a-8e1d-439c-98b4-bd9fac2ecec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:29.294878 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212d87a-8e1d-439c-98b4-bd9fac2ecec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:29.295331 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212d87a-8e1d-439c-98b4-bd9fac2ecec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:29.295760 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212d87a-8e1d-439c-98b4-bd9fac2ecec1 admin admin] PATH_INFO: `/v3/projects/043ad101133647c0b8f3ac10b2e9a5ce/users/40f6dd61918c412183877995b86243dd/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:29.331675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212d87a-8e1d-439c-98b4-bd9fac2ecec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=043ad101133647c0b8f3ac10b2e9a5ce, user_id=40f6dd61918c412183877995b86243dd, role_id=c89b1b3b1b1a4b67ab5c6c805014dd6d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:29.337461 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212d87a-8e1d-439c-98b4-bd9fac2ecec1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:29.366959 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1423/2842] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:29 2022] PUT /identity/v3/projects/043ad101133647c0b8f3ac10b2e9a5ce/users/40f6dd61918c412183877995b86243dd/roles/c89b1b3b1b1a4b67ab5c6c805014dd6d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:29.377647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c6c84-4b95-4981-8bb0-a01d078d7f47 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:29.377862 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c6c84-4b95-4981-8bb0-a01d078d7f47 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:29.378148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c6c84-4b95-4981-8bb0-a01d078d7f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:29.439790 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-218c6c84-4b95-4981-8bb0-a01d078d7f47 None None] MFA Rules not processed for user `40f6dd61918c412183877995b86243dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:29.441026 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-218c6c84-4b95-4981-8bb0-a01d078d7f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:29.587583 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1420/2843] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:29.592296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212d87a-8e1d-439c-98b4-bd9fac2ecec1 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:29.732410 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212d87a-8e1d-439c-98b4-bd9fac2ecec1 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:29.756361 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d79057-c38b-4138-83e1-5947226a5f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:29.758034 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d79057-c38b-4138-83e1-5947226a5f94 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:29.758333 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d79057-c38b-4138-83e1-5947226a5f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:29.758617 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d79057-c38b-4138-83e1-5947226a5f94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:29.759335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d79057-c38b-4138-83e1-5947226a5f94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:29.762335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d79057-c38b-4138-83e1-5947226a5f94 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:29.839340 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1424/2844] 10.222.0.79 () {60 vars in 1152 bytes} [Mon Jul 4 10:17:29 2022] POST /identity/v3/projects => generated 377 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:29.843363 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c6c84-4b95-4981-8bb0-a01d078d7f47 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:29.867678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c6c84-4b95-4981-8bb0-a01d078d7f47 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:29.877954 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2067a5-f4d7-4194-b126-40399d28fd01 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:29.878171 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2067a5-f4d7-4194-b126-40399d28fd01 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:29.878455 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2067a5-f4d7-4194-b126-40399d28fd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:29.933280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5b2067a5-f4d7-4194-b126-40399d28fd01 None None] MFA Rules not processed for user `2321028b4d4f4734afd1ce674931a4d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:29.934603 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5b2067a5-f4d7-4194-b126-40399d28fd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:29.957208 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:29.991843 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:30.029262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacc644-1090-4923-96a5-84ed12bc81c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:30.030190 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacc644-1090-4923-96a5-84ed12bc81c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:30.030479 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacc644-1090-4923-96a5-84ed12bc81c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:30.030758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacc644-1090-4923-96a5-84ed12bc81c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:30.035067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacc644-1090-4923-96a5-84ed12bc81c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:30.037096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacc644-1090-4923-96a5-84ed12bc81c2 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:30.088070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1425/2845] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:30.091166 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1421/2846] 10.222.0.79 () {60 vars in 1143 bytes} [Mon Jul 4 10:17:29 2022] POST /identity/v3/users => generated 295 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 10:17:30.095355 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2067a5-f4d7-4194-b126-40399d28fd01 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:30.109984 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2067a5-f4d7-4194-b126-40399d28fd01 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:30.116280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51853c-45f6-4922-bec4-695fddae42bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:30.116998 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51853c-45f6-4922-bec4-695fddae42bb admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:30.117264 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51853c-45f6-4922-bec4-695fddae42bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:30.118192 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51853c-45f6-4922-bec4-695fddae42bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:30.119101 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51853c-45f6-4922-bec4-695fddae42bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:30.120759 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51853c-45f6-4922-bec4-695fddae42bb admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:30.139757 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1426/2847] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:30.145229 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacc644-1090-4923-96a5-84ed12bc81c2 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:30.156878 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacc644-1090-4923-96a5-84ed12bc81c2 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:30.163134 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560638e-4241-453c-b1d7-c3deeee247f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:30.164093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560638e-4241-453c-b1d7-c3deeee247f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:30.164393 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560638e-4241-453c-b1d7-c3deeee247f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:30.164640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560638e-4241-453c-b1d7-c3deeee247f8 admin admin] PATH_INFO: `/v3/projects/e005b8685e16459bb4abd47d6cd81443/users/c668291c82654f4886a875040e423123/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:30.234172 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560638e-4241-453c-b1d7-c3deeee247f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e005b8685e16459bb4abd47d6cd81443, user_id=c668291c82654f4886a875040e423123, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:30.237012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560638e-4241-453c-b1d7-c3deeee247f8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:30.266359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1422/2848] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:30 2022] PUT /identity/v3/projects/e005b8685e16459bb4abd47d6cd81443/users/c668291c82654f4886a875040e423123/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:30.270677 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51853c-45f6-4922-bec4-695fddae42bb admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:30.289966 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51853c-45f6-4922-bec4-695fddae42bb admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:30.364615 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d96a2-6bfe-4291-a25a-72c9e93200a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:30.365414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d96a2-6bfe-4291-a25a-72c9e93200a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:30.365828 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d96a2-6bfe-4291-a25a-72c9e93200a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:30.366212 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d96a2-6bfe-4291-a25a-72c9e93200a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:30.367020 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d96a2-6bfe-4291-a25a-72c9e93200a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:30.368782 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d96a2-6bfe-4291-a25a-72c9e93200a5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:30.375056 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d411c84-bec5-4725-9116-2b06f6d4f690 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:30.375555 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d411c84-bec5-4725-9116-2b06f6d4f690 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:30.375937 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d411c84-bec5-4725-9116-2b06f6d4f690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:30.412322 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1427/2849] 10.222.0.79 () {58 vars in 1121 bytes} [Mon Jul 4 10:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 10:17:30.416944 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d96a2-6bfe-4291-a25a-72c9e93200a5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:30.426891 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-6d411c84-bec5-4725-9116-2b06f6d4f690 None None] MFA Rules not processed for user `ace3c4ac54514056b9a48abf9a3cb64a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:30.428486 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6d411c84-bec5-4725-9116-2b06f6d4f690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:30.428606 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d96a2-6bfe-4291-a25a-72c9e93200a5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:30.434292 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fa9f7-9a99-4463-8dc0-f30e3286806f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:30.435211 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fa9f7-9a99-4463-8dc0-f30e3286806f admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:30.435497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fa9f7-9a99-4463-8dc0-f30e3286806f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:30.435774 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fa9f7-9a99-4463-8dc0-f30e3286806f admin admin] PATH_INFO: `/v3/projects/e005b8685e16459bb4abd47d6cd81443/users/c668291c82654f4886a875040e423123/roles/a3de9df253294d069dc1867fa26600c3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:30.439562 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fa9f7-9a99-4463-8dc0-f30e3286806f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e005b8685e16459bb4abd47d6cd81443, user_id=c668291c82654f4886a875040e423123, role_id=a3de9df253294d069dc1867fa26600c3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:30.458815 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fa9f7-9a99-4463-8dc0-f30e3286806f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:30.500147 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1428/2850] 10.222.0.79 () {60 vars in 1482 bytes} [Mon Jul 4 10:17:30 2022] PUT /identity/v3/projects/e005b8685e16459bb4abd47d6cd81443/users/c668291c82654f4886a875040e423123/roles/a3de9df253294d069dc1867fa26600c3 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:30.511832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a5d34-8007-42cf-b391-69c52d944e99 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:30.512075 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a5d34-8007-42cf-b391-69c52d944e99 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:30.512291 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a5d34-8007-42cf-b391-69c52d944e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:30.553574 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1423/2851] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:30.570570 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d411c84-bec5-4725-9116-2b06f6d4f690 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:30.572528 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6d411c84-bec5-4725-9116-2b06f6d4f690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:30.587843 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-554a5d34-8007-42cf-b391-69c52d944e99 None None] MFA Rules not processed for user `c668291c82654f4886a875040e423123`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:30.589093 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-554a5d34-8007-42cf-b391-69c52d944e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:30.716074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1429/2852] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:30.725572 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d411c84-bec5-4725-9116-2b06f6d4f690 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:30.736073 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a5d34-8007-42cf-b391-69c52d944e99 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:30.738335 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-554a5d34-8007-42cf-b391-69c52d944e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:30.786153 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:30.806277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd6fc7-00c5-4d1b-9d6d-0bede9ce1a44 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef685a17c804e2bbfea1e07cfdf5cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:30.808744 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd6fc7-00c5-4d1b-9d6d-0bede9ce1a44 service glance] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:30.809045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd6fc7-00c5-4d1b-9d6d-0bede9ce1a44 service glance] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:30.809328 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd6fc7-00c5-4d1b-9d6d-0bede9ce1a44 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:30.811593 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-74bd6fc7-00c5-4d1b-9d6d-0bede9ce1a44 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:30.822174 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:30.869074 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a5d34-8007-42cf-b391-69c52d944e99 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:30.880752 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7fe7b-baea-4bd8-84ae-1a49f70183b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:30.881438 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7fe7b-baea-4bd8-84ae-1a49f70183b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:30.882363 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7fe7b-baea-4bd8-84ae-1a49f70183b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:30.882853 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7fe7b-baea-4bd8-84ae-1a49f70183b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:30.922055 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd6fc7-00c5-4d1b-9d6d-0bede9ce1a44 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:30.923445 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd6fc7-00c5-4d1b-9d6d-0bede9ce1a44 service glance] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:30.941760 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1424/2853] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 10:17:31.003003 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7fe7b-baea-4bd8-84ae-1a49f70183b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:31.006141 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7fe7b-baea-4bd8-84ae-1a49f70183b3 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:31.025888 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1430/2854] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:17:31.114046 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fed6f8-e85f-4571-8f7e-3d1551dc6310 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:31.114348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fed6f8-e85f-4571-8f7e-3d1551dc6310 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:31.114632 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fed6f8-e85f-4571-8f7e-3d1551dc6310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:31.142896 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c7fed6f8-e85f-4571-8f7e-3d1551dc6310 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:31.144640 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c7fed6f8-e85f-4571-8f7e-3d1551dc6310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:31.231428 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1425/2855] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:17:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:31.246122 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7fe7b-baea-4bd8-84ae-1a49f70183b3 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:31.248136 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-21f7fe7b-baea-4bd8-84ae-1a49f70183b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:31.423663 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7fe7b-baea-4bd8-84ae-1a49f70183b3 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:31.430807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7316b-cea9-4eef-8baa-865501e88fa0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:31.431485 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7316b-cea9-4eef-8baa-865501e88fa0 service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:31.431766 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7316b-cea9-4eef-8baa-865501e88fa0 service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:31.432042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7316b-cea9-4eef-8baa-865501e88fa0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:31.445210 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7316b-cea9-4eef-8baa-865501e88fa0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:31.446990 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7316b-cea9-4eef-8baa-865501e88fa0 service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:31.461791 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1431/2856] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:17:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 10:17:32.733930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fed6f8-e85f-4571-8f7e-3d1551dc6310 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:32.736154 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c7fed6f8-e85f-4571-8f7e-3d1551dc6310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:32.758176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fed6f8-e85f-4571-8f7e-3d1551dc6310 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:32.779800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc3961-af1d-4394-a8d5-17d0dac05a18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:32.780525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc3961-af1d-4394-a8d5-17d0dac05a18 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:32.780811 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc3961-af1d-4394-a8d5-17d0dac05a18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:32.781096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc3961-af1d-4394-a8d5-17d0dac05a18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:32.889177 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc3961-af1d-4394-a8d5-17d0dac05a18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:32.892294 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc3961-af1d-4394-a8d5-17d0dac05a18 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:32.905379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1426/2857] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:17:34.090279 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7316b-cea9-4eef-8baa-865501e88fa0 service swift] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:34.100462 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7316b-cea9-4eef-8baa-865501e88fa0 service swift] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:34.105156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f7342-2d84-4334-8bd0-14ba36b0af80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a4af83902d4370a832ed3ea3861b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:34.105870 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f7342-2d84-4334-8bd0-14ba36b0af80 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:34.106245 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f7342-2d84-4334-8bd0-14ba36b0af80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:34.106442 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f7342-2d84-4334-8bd0-14ba36b0af80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:34.120535 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f7342-2d84-4334-8bd0-14ba36b0af80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:34.147991 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f7342-2d84-4334-8bd0-14ba36b0af80 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:34.183048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1432/2858] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:17:38.608358 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc3961-af1d-4394-a8d5-17d0dac05a18 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:38.670824 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:38.688525 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc3961-af1d-4394-a8d5-17d0dac05a18 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:38.693904 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:38.694800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:38.695078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:38.695353 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] PATH_INFO: `/v3/users/656ac2482c094050b48a529ee5e84bd0` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:38.696594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=656ac2482c094050b48a529ee5e84bd0)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:38.698758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:38.760527 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '656ac2482c094050b48a529ee5e84bd0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:38.761271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '656ac2482c094050b48a529ee5e84bd0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:38.772194 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '656ac2482c094050b48a529ee5e84bd0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:38.789072 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '656ac2482c094050b48a529ee5e84bd0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:38.790093 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '656ac2482c094050b48a529ee5e84bd0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:38.804931 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1427/2859] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:17:38 2022] DELETE /identity/v3/users/656ac2482c094050b48a529ee5e84bd0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:38.809244 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc62a20-e2f1-4e87-ac46-90953a56a8d8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:38.809785 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc62a20-e2f1-4e87-ac46-90953a56a8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:38.809916 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc62a20-e2f1-4e87-ac46-90953a56a8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:38.830534 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ebc62a20-e2f1-4e87-ac46-90953a56a8d8 None None] MFA Rules not processed for user `5875b90a72824e1280ee921aaccb7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:38.831768 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ebc62a20-e2f1-4e87-ac46-90953a56a8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:38.887118 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:38.909559 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1433/2860] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:38.921272 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:38.922976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:38.992948 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69486c0c-81e0-451c-a162-f96d303283f9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:38.998788 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a25e1-17c7-41b0-a62e-6193f30884e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:38.999461 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a25e1-17c7-41b0-a62e-6193f30884e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:38.999743 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a25e1-17c7-41b0-a62e-6193f30884e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:39.000023 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a25e1-17c7-41b0-a62e-6193f30884e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:39.011311 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a25e1-17c7-41b0-a62e-6193f30884e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:39.012796 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a25e1-17c7-41b0-a62e-6193f30884e4 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:39.027768 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1428/2861] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:17:39.349307 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc62a20-e2f1-4e87-ac46-90953a56a8d8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:39.350806 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ebc62a20-e2f1-4e87-ac46-90953a56a8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:39.386296 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc62a20-e2f1-4e87-ac46-90953a56a8d8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:39.394316 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:39.395183 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:39.395518 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:39.395912 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] PATH_INFO: `/v3/projects/3d5813e628834a86b353d67cbc6d8cec` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:39.398379 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d5813e628834a86b353d67cbc6d8cec)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:39.401025 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:39.466164 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d5813e628834a86b353d67cbc6d8cec', 'request_id': 'req-bfbbeb79-7852-4c1a-b492-ec58756d0acf'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:39.474739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d5813e628834a86b353d67cbc6d8cec', 'request_id': 'req-bfbbeb79-7852-4c1a-b492-ec58756d0acf'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:39.485632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d5813e628834a86b353d67cbc6d8cec', 'request_id': 'req-bfbbeb79-7852-4c1a-b492-ec58756d0acf'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:39.509042 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] The token cache is being invalidate because project 3d5813e628834a86b353d67cbc6d8cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:17:39.509377 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:39.513838 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:39.516030 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1434/2862] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:17:39 2022] DELETE /identity/v3/projects/3d5813e628834a86b353d67cbc6d8cec => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:39.520795 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a25e1-17c7-41b0-a62e-6193f30884e4 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:39.533434 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a25e1-17c7-41b0-a62e-6193f30884e4 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:39.612953 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:39.613914 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:39.614201 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:39.614559 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] PATH_INFO: `/v3/users/40f6dd61918c412183877995b86243dd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:39.615771 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40f6dd61918c412183877995b86243dd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:39.617782 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:39.631283 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:39.676635 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40f6dd61918c412183877995b86243dd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:39.677316 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40f6dd61918c412183877995b86243dd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:39.686452 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40f6dd61918c412183877995b86243dd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:39.702077 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40f6dd61918c412183877995b86243dd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:39.702900 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40f6dd61918c412183877995b86243dd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:39.718701 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1429/2863] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:17:39 2022] DELETE /identity/v3/users/40f6dd61918c412183877995b86243dd => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:40.029113 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:40.031287 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:40.049119 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbeb79-7852-4c1a-b492-ec58756d0acf admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:40.117241 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:40.121209 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:40.121501 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:40.121813 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] PATH_INFO: `/v3/projects/043ad101133647c0b8f3ac10b2e9a5ce` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:40.129043 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] RBAC: Authorizing `identity:delete_project(project_id=043ad101133647c0b8f3ac10b2e9a5ce)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:40.136226 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:40.200900 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '043ad101133647c0b8f3ac10b2e9a5ce', 'request_id': 'req-ff0cb09e-8f91-4e61-874f-4a74113ea645'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:40.208537 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '043ad101133647c0b8f3ac10b2e9a5ce', 'request_id': 'req-ff0cb09e-8f91-4e61-874f-4a74113ea645'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:40.215762 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '043ad101133647c0b8f3ac10b2e9a5ce', 'request_id': 'req-ff0cb09e-8f91-4e61-874f-4a74113ea645'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:40.233058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] The token cache is being invalidate because project 043ad101133647c0b8f3ac10b2e9a5ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:17:40.233525 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:40.234299 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:40.235982 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1435/2864] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:17:40 2022] DELETE /identity/v3/projects/043ad101133647c0b8f3ac10b2e9a5ce => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:40.240019 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:40.241652 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:40.255908 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474244a3-2c8d-4e17-bcee-538d24169833 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:40.318747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4382ecb-58e0-4e43-a587-e37e6ccd2ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:40.319645 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4382ecb-58e0-4e43-a587-e37e6ccd2ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:40.319923 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4382ecb-58e0-4e43-a587-e37e6ccd2ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:40.320195 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4382ecb-58e0-4e43-a587-e37e6ccd2ded admin admin] PATH_INFO: `/v3/users/c668291c82654f4886a875040e423123` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:40.321734 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4382ecb-58e0-4e43-a587-e37e6ccd2ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=c668291c82654f4886a875040e423123)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:40.324109 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4382ecb-58e0-4e43-a587-e37e6ccd2ded admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:40.383416 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a4382ecb-58e0-4e43-a587-e37e6ccd2ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c668291c82654f4886a875040e423123'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:40.384103 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a4382ecb-58e0-4e43-a587-e37e6ccd2ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c668291c82654f4886a875040e423123'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:40.395029 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a4382ecb-58e0-4e43-a587-e37e6ccd2ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c668291c82654f4886a875040e423123'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:40.409315 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a4382ecb-58e0-4e43-a587-e37e6ccd2ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c668291c82654f4886a875040e423123'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:40.410219 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a4382ecb-58e0-4e43-a587-e37e6ccd2ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c668291c82654f4886a875040e423123'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:40.424531 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1430/2865] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:17:40 2022] DELETE /identity/v3/users/c668291c82654f4886a875040e423123 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:40.798904 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:40.800520 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:40.818423 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:40.839910 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cb09e-8f91-4e61-874f-4a74113ea645 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:40.908170 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc0bfb-004e-4429-8896-0e65cd99289b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:40.909128 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc0bfb-004e-4429-8896-0e65cd99289b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:40.909408 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc0bfb-004e-4429-8896-0e65cd99289b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:40.909708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc0bfb-004e-4429-8896-0e65cd99289b admin admin] PATH_INFO: `/v3/projects/e005b8685e16459bb4abd47d6cd81443` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:40.913223 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc0bfb-004e-4429-8896-0e65cd99289b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e005b8685e16459bb4abd47d6cd81443)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:40.915327 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc0bfb-004e-4429-8896-0e65cd99289b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:40.978166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3adc0bfb-004e-4429-8896-0e65cd99289b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e005b8685e16459bb4abd47d6cd81443', 'request_id': 'req-3adc0bfb-004e-4429-8896-0e65cd99289b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:40.985031 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3adc0bfb-004e-4429-8896-0e65cd99289b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e005b8685e16459bb4abd47d6cd81443', 'request_id': 'req-3adc0bfb-004e-4429-8896-0e65cd99289b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:40.992328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3adc0bfb-004e-4429-8896-0e65cd99289b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e005b8685e16459bb4abd47d6cd81443', 'request_id': 'req-3adc0bfb-004e-4429-8896-0e65cd99289b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:41.011546 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3adc0bfb-004e-4429-8896-0e65cd99289b admin admin] The token cache is being invalidate because project e005b8685e16459bb4abd47d6cd81443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:17:41.011875 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3adc0bfb-004e-4429-8896-0e65cd99289b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:41.012648 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3adc0bfb-004e-4429-8896-0e65cd99289b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:17:41.014578 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1436/2866] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:17:40 2022] DELETE /identity/v3/projects/e005b8685e16459bb4abd47d6cd81443 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:17:41.957041 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272ab44-3d13-4514-872d-986a06d8f12a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:41.957654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272ab44-3d13-4514-872d-986a06d8f12a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:41.957654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272ab44-3d13-4514-872d-986a06d8f12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:41.977016 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f272ab44-3d13-4514-872d-986a06d8f12a None None] MFA Rules not processed for user `2264f97424ae45ffa5770a66e95897c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:41.978341 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f272ab44-3d13-4514-872d-986a06d8f12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:41.982421 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:42.037305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1431/2867] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:17:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:43.808735 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1db89-71b0-4707-a342-5ba66c770d0c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:43.809248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1db89-71b0-4707-a342-5ba66c770d0c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:43.809344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1db89-71b0-4707-a342-5ba66c770d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:43.827961 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a5b1db89-71b0-4707-a342-5ba66c770d0c None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:17:43.829174 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a5b1db89-71b0-4707-a342-5ba66c770d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:43.887414 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1437/2868] 10.222.0.79 () {58 vars in 1009 bytes} [Mon Jul 4 10:17:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:17:43.900328 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272ab44-3d13-4514-872d-986a06d8f12a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:43.901800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f272ab44-3d13-4514-872d-986a06d8f12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:43.974727 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272ab44-3d13-4514-872d-986a06d8f12a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:43.980080 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd495cee-66ac-4caf-badb-b2fd4303553a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:43.980739 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495cee-66ac-4caf-badb-b2fd4303553a service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:43.981027 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495cee-66ac-4caf-badb-b2fd4303553a service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:43.981305 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495cee-66ac-4caf-badb-b2fd4303553a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:43.990553 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd495cee-66ac-4caf-badb-b2fd4303553a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:43.992191 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd495cee-66ac-4caf-badb-b2fd4303553a service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:44.000798 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1432/2869] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:17:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 10:17:53.879828 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:53.997264 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=53 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:17:54.317497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1db89-71b0-4707-a342-5ba66c770d0c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:17:54.319290 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a5b1db89-71b0-4707-a342-5ba66c770d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:54.378435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1db89-71b0-4707-a342-5ba66c770d0c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:17:54.383328 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a52da9-bca8-4439-bc8d-0b8774658470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:17:54.383992 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a52da9-bca8-4439-bc8d-0b8774658470 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:17:54.384250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a52da9-bca8-4439-bc8d-0b8774658470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:17:54.384560 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a52da9-bca8-4439-bc8d-0b8774658470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:17:54.386649 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-69a52da9-bca8-4439-bc8d-0b8774658470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:17:54.458865 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a52da9-bca8-4439-bc8d-0b8774658470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:17:54.460726 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a52da9-bca8-4439-bc8d-0b8774658470 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:17:54.472401 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1438/2870] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:17:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:18:04.468502 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=50 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:18:09.949176 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd495cee-66ac-4caf-badb-b2fd4303553a service swift] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:18:09.959012 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd495cee-66ac-4caf-badb-b2fd4303553a service swift] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:18:09.963406 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e581ca1-dd3e-49ba-9c16-5f81d43806e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:18:09.964044 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e581ca1-dd3e-49ba-9c16-5f81d43806e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:18:09.964339 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e581ca1-dd3e-49ba-9c16-5f81d43806e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:18:09.964851 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e581ca1-dd3e-49ba-9c16-5f81d43806e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:18:09.967096 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4e581ca1-dd3e-49ba-9c16-5f81d43806e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:18:10.036480 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e581ca1-dd3e-49ba-9c16-5f81d43806e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:18:10.037983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e581ca1-dd3e-49ba-9c16-5f81d43806e2 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:18:10.048348 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1433/2871] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:18:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:18:20.043606 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=30 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:18:37.112007 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a52da9-bca8-4439-bc8d-0b8774658470 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:18:37.121158 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a52da9-bca8-4439-bc8d-0b8774658470 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:18:37.125018 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98049384-a6af-4e84-81bd-235d568858dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:18:37.125708 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98049384-a6af-4e84-81bd-235d568858dc service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:18:37.125993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98049384-a6af-4e84-81bd-235d568858dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:18:37.126271 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98049384-a6af-4e84-81bd-235d568858dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:18:37.128058 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-98049384-a6af-4e84-81bd-235d568858dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:18:37.207174 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98049384-a6af-4e84-81bd-235d568858dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:18:37.208597 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98049384-a6af-4e84-81bd-235d568858dc service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:18:37.221072 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1439/2872] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:18:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:18:38.520471 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e581ca1-dd3e-49ba-9c16-5f81d43806e2 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:18:38.534007 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e581ca1-dd3e-49ba-9c16-5f81d43806e2 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:18:38.538656 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23bbab-9858-4c95-be74-da09610f2fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:18:38.539293 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23bbab-9858-4c95-be74-da09610f2fea service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:18:38.539579 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23bbab-9858-4c95-be74-da09610f2fea service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:18:38.539865 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23bbab-9858-4c95-be74-da09610f2fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:18:38.541793 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ae23bbab-9858-4c95-be74-da09610f2fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:18:38.557175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23bbab-9858-4c95-be74-da09610f2fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:18:38.558801 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23bbab-9858-4c95-be74-da09610f2fea service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:18:38.571225 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1434/2873] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:18:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:18:47.216459 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=34 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:18:48.565291 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=6 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:19:22.438435 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98049384-a6af-4e84-81bd-235d568858dc service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:19:22.451681 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98049384-a6af-4e84-81bd-235d568858dc service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:19:22.468886 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905aea68-a5a4-462d-8bd9-d0d25a406347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:19:22.472091 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905aea68-a5a4-462d-8bd9-d0d25a406347 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:19:22.474872 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905aea68-a5a4-462d-8bd9-d0d25a406347 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:19:22.475689 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905aea68-a5a4-462d-8bd9-d0d25a406347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:19:22.480174 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-905aea68-a5a4-462d-8bd9-d0d25a406347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:19:22.498221 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905aea68-a5a4-462d-8bd9-d0d25a406347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:19:22.499262 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905aea68-a5a4-462d-8bd9-d0d25a406347 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:19:22.512109 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1440/2874] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:19:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:19:30.080659 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f6e3f-65c7-42c7-84f3-05fe29dbe667 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:19:30.081165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f6e3f-65c7-42c7-84f3-05fe29dbe667 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:19:30.081275 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f6e3f-65c7-42c7-84f3-05fe29dbe667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:19:30.100252 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-585f6e3f-65c7-42c7-84f3-05fe29dbe667 None None] MFA Rules not processed for user `5bdf239fa2bc4f1b9e484f916c438c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:19:30.101736 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-585f6e3f-65c7-42c7-84f3-05fe29dbe667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:19:30.109450 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1435/2875] 10.222.0.79 () {58 vars in 967 bytes} [Mon Jul 4 10:19:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:19:30.154104 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905aea68-a5a4-462d-8bd9-d0d25a406347 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:19:30.167156 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905aea68-a5a4-462d-8bd9-d0d25a406347 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:19:30.172388 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfe57a-d1e4-4273-bad7-5bd9561ec563 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ec37893ec8c480990c17cb03cb0ca19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:19:30.173025 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfe57a-d1e4-4273-bad7-5bd9561ec563 service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:19:30.173313 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfe57a-d1e4-4273-bad7-5bd9561ec563 service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:19:30.173593 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfe57a-d1e4-4273-bad7-5bd9561ec563 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:19:30.182269 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfe57a-d1e4-4273-bad7-5bd9561ec563 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:19:30.183873 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfe57a-d1e4-4273-bad7-5bd9561ec563 service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:19:30.192148 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1441/2876] 10.222.0.79 () {60 vars in 1384 bytes} [Mon Jul 4 10:19:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 10:19:33.725601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f6e3f-65c7-42c7-84f3-05fe29dbe667 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:19:33.727309 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-585f6e3f-65c7-42c7-84f3-05fe29dbe667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:19:33.805922 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f6e3f-65c7-42c7-84f3-05fe29dbe667 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:19:33.810725 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977ac68-d72d-48bc-801f-c5d45cc83547 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:19:33.811475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977ac68-d72d-48bc-801f-c5d45cc83547 service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:19:33.811827 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977ac68-d72d-48bc-801f-c5d45cc83547 service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:19:33.812115 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977ac68-d72d-48bc-801f-c5d45cc83547 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:19:33.813983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5977ac68-d72d-48bc-801f-c5d45cc83547 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:19:33.826316 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977ac68-d72d-48bc-801f-c5d45cc83547 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:19:33.827921 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977ac68-d72d-48bc-801f-c5d45cc83547 service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:19:33.838892 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1436/2877] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:19:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:19:36.760674 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfe57a-d1e4-4273-bad7-5bd9561ec563 service swift] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:19:36.762345 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-dfcfe57a-d1e4-4273-bad7-5bd9561ec563 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:19:36.841653 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfe57a-d1e4-4273-bad7-5bd9561ec563 service swift] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:19:36.845492 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:19:36.846484 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:19:36.846793 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:19:36.847079 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] PATH_INFO: `/v3/users/2264f97424ae45ffa5770a66e95897c7` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:19:36.848207 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2264f97424ae45ffa5770a66e95897c7)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:19:36.850194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:19:36.902632 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2264f97424ae45ffa5770a66e95897c7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:36.903538 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2264f97424ae45ffa5770a66e95897c7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:36.912712 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2264f97424ae45ffa5770a66e95897c7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:36.925224 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2264f97424ae45ffa5770a66e95897c7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:36.925945 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2264f97424ae45ffa5770a66e95897c7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:36.938483 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1442/2878] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:19:36 2022] DELETE /identity/v3/users/2264f97424ae45ffa5770a66e95897c7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:19:37.130899 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977ac68-d72d-48bc-801f-c5d45cc83547 service placement] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:19:37.144180 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977ac68-d72d-48bc-801f-c5d45cc83547 service placement] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:19:37.204095 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:19:37.204970 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:19:37.205259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:19:37.205533 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] PATH_INFO: `/v3/projects/09b54b24115e4074a79f93c255d4c0ca` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:19:37.206800 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b54b24115e4074a79f93c255d4c0ca)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:19:37.208758 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:19:37.259019 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b54b24115e4074a79f93c255d4c0ca', 'request_id': 'req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.265475 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b54b24115e4074a79f93c255d4c0ca', 'request_id': 'req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.271654 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b54b24115e4074a79f93c255d4c0ca', 'request_id': 'req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.288228 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] The token cache is being invalidate because project 09b54b24115e4074a79f93c255d4c0ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:19:37.288501 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.289144 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.290983 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1437/2879] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:19:37 2022] DELETE /identity/v3/projects/09b54b24115e4074a79f93c255d4c0ca => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:19:37.295187 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:19:37.296725 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:19:37.310458 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacd618-0c1b-462c-aea7-9a4d011c47a9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:19:37.361060 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:19:37.372407 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:19:37.373253 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:19:37.373543 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:19:37.373901 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] PATH_INFO: `/v3/users/ace3c4ac54514056b9a48abf9a3cb64a` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:19:37.375356 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace3c4ac54514056b9a48abf9a3cb64a)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:19:37.377607 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:19:37.430320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace3c4ac54514056b9a48abf9a3cb64a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.431035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace3c4ac54514056b9a48abf9a3cb64a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.440698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace3c4ac54514056b9a48abf9a3cb64a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.452890 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace3c4ac54514056b9a48abf9a3cb64a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.453519 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace3c4ac54514056b9a48abf9a3cb64a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.465980 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1443/2880] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:19:37 2022] DELETE /identity/v3/users/ace3c4ac54514056b9a48abf9a3cb64a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:19:37.647493 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:19:37.649067 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:19:37.662175 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72d17f-6bfb-4b12-9b1a-b4588ea113b8 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:19:37.673078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:19:37.730620 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2a66c-be61-4279-b2a0-30986b5289d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:19:37.731502 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2a66c-be61-4279-b2a0-30986b5289d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:19:37.731806 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2a66c-be61-4279-b2a0-30986b5289d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:19:37.732089 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2a66c-be61-4279-b2a0-30986b5289d6 admin admin] PATH_INFO: `/v3/projects/8a785e9fae27434bbd0cac2396834fe8` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:19:37.733322 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2a66c-be61-4279-b2a0-30986b5289d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a785e9fae27434bbd0cac2396834fe8)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:19:37.735304 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2a66c-be61-4279-b2a0-30986b5289d6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:19:37.786165 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-0de2a66c-be61-4279-b2a0-30986b5289d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a785e9fae27434bbd0cac2396834fe8', 'request_id': 'req-0de2a66c-be61-4279-b2a0-30986b5289d6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.798379 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-0de2a66c-be61-4279-b2a0-30986b5289d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a785e9fae27434bbd0cac2396834fe8', 'request_id': 'req-0de2a66c-be61-4279-b2a0-30986b5289d6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.804431 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-0de2a66c-be61-4279-b2a0-30986b5289d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a785e9fae27434bbd0cac2396834fe8', 'request_id': 'req-0de2a66c-be61-4279-b2a0-30986b5289d6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.819940 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-0de2a66c-be61-4279-b2a0-30986b5289d6 admin admin] The token cache is being invalidate because project 8a785e9fae27434bbd0cac2396834fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:19:37.820229 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-0de2a66c-be61-4279-b2a0-30986b5289d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.820890 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-0de2a66c-be61-4279-b2a0-30986b5289d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:19:37.823185 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1438/2881] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:19:37 2022] DELETE /identity/v3/projects/8a785e9fae27434bbd0cac2396834fe8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:19:47.462548 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:19:47.816556 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:19:56.793788 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:19:56.795193 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:19:56.882076 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab43b6e-e80f-4708-95c4-c4a96ef16ee0 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:19:56.947440 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41788b95-9a03-4ef4-9690-948366058e44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:19:56.948125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41788b95-9a03-4ef4-9690-948366058e44 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:19:56.948399 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41788b95-9a03-4ef4-9690-948366058e44 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:19:56.949214 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41788b95-9a03-4ef4-9690-948366058e44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:19:56.950490 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-41788b95-9a03-4ef4-9690-948366058e44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:19:56.964448 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41788b95-9a03-4ef4-9690-948366058e44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:19:56.966488 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41788b95-9a03-4ef4-9690-948366058e44 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:19:56.978993 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1444/2882] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:19:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:19:57.862227 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb4e7c-f975-4f85-81d2-3db84c77a92e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:19:57.862754 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb4e7c-f975-4f85-81d2-3db84c77a92e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:19:57.862754 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb4e7c-f975-4f85-81d2-3db84c77a92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:19:57.883131 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c7cb4e7c-f975-4f85-81d2-3db84c77a92e None None] MFA Rules not processed for user `2321028b4d4f4734afd1ce674931a4d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:19:57.884255 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c7cb4e7c-f975-4f85-81d2-3db84c77a92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:19:57.944882 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1439/2883] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:19:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:19:58.278545 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41788b95-9a03-4ef4-9690-948366058e44 service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:19:58.282024 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-41788b95-9a03-4ef4-9690-948366058e44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:19:58.342982 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41788b95-9a03-4ef4-9690-948366058e44 service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:19:58.347468 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806e43c-0205-445b-819b-e4815d20d98b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:19:58.348112 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806e43c-0205-445b-819b-e4815d20d98b service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:19:58.348390 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806e43c-0205-445b-819b-e4815d20d98b service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:19:58.348656 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806e43c-0205-445b-819b-e4815d20d98b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:19:58.350814 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0806e43c-0205-445b-819b-e4815d20d98b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:19:58.361008 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806e43c-0205-445b-819b-e4815d20d98b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:19:58.362479 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806e43c-0205-445b-819b-e4815d20d98b service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:19:58.370761 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1445/2884] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:19:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:20:00.882913 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb4e7c-f975-4f85-81d2-3db84c77a92e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:20:00.884626 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c7cb4e7c-f975-4f85-81d2-3db84c77a92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:20:00.959230 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb4e7c-f975-4f85-81d2-3db84c77a92e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:20:00.965544 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0c1d8-978f-4383-b119-4cfd7ed27cdc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:20:00.966243 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0c1d8-978f-4383-b119-4cfd7ed27cdc service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:20:00.966527 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0c1d8-978f-4383-b119-4cfd7ed27cdc service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:20:00.966802 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0c1d8-978f-4383-b119-4cfd7ed27cdc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:20:00.968860 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-29c0c1d8-978f-4383-b119-4cfd7ed27cdc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:20:00.982214 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0c1d8-978f-4383-b119-4cfd7ed27cdc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:20:00.983935 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0c1d8-978f-4383-b119-4cfd7ed27cdc service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:20:00.995193 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1440/2885] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:20:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:20:08.367285 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=88 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:20:10.991578 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=57 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:20:15.055804 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806e43c-0205-445b-819b-e4815d20d98b service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:20:15.057304 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0806e43c-0205-445b-819b-e4815d20d98b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:20:15.076028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806e43c-0205-445b-819b-e4815d20d98b service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:20:15.082028 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40470125-cef3-4836-9ff7-f0129c7cc902 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:20:15.082680 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40470125-cef3-4836-9ff7-f0129c7cc902 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:20:15.082962 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40470125-cef3-4836-9ff7-f0129c7cc902 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:20:15.083704 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40470125-cef3-4836-9ff7-f0129c7cc902 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:20:15.085698 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-40470125-cef3-4836-9ff7-f0129c7cc902 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:20:15.098622 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40470125-cef3-4836-9ff7-f0129c7cc902 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:20:15.100600 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40470125-cef3-4836-9ff7-f0129c7cc902 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:20:15.111662 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1446/2886] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:20:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:20:25.107329 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:21:53.546709 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef24e1-272d-4ac3-a3e7-41d7ba9c463c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:21:53.547277 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef24e1-272d-4ac3-a3e7-41d7ba9c463c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:21:53.547394 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef24e1-272d-4ac3-a3e7-41d7ba9c463c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:21:53.577863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-66ef24e1-272d-4ac3-a3e7-41d7ba9c463c None None] MFA Rules not processed for user `2321028b4d4f4734afd1ce674931a4d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 10:21:53.579078 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-66ef24e1-272d-4ac3-a3e7-41d7ba9c463c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:21:53.606462 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1441/2887] 10.222.0.79 () {58 vars in 957 bytes} [Mon Jul 4 10:21:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 10:21:53.932071 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40470125-cef3-4836-9ff7-f0129c7cc902 service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:21:53.974194 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40470125-cef3-4836-9ff7-f0129c7cc902 service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:21:53.979090 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55505033-2e46-4268-821a-7e0a934efea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:21:53.979737 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55505033-2e46-4268-821a-7e0a934efea9 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:21:53.980023 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55505033-2e46-4268-821a-7e0a934efea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:21:53.980290 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55505033-2e46-4268-821a-7e0a934efea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:21:53.982497 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-55505033-2e46-4268-821a-7e0a934efea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:21:54.006166 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55505033-2e46-4268-821a-7e0a934efea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:21:54.007621 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55505033-2e46-4268-821a-7e0a934efea9 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:21:54.016378 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1447/2888] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:21:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:21:54.940964 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef24e1-272d-4ac3-a3e7-41d7ba9c463c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:21:54.942595 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-66ef24e1-272d-4ac3-a3e7-41d7ba9c463c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:21:54.991494 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef24e1-272d-4ac3-a3e7-41d7ba9c463c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:21:54.998164 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa667bd9-e2ca-4481-ac3d-1c34078d7dcc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:21:54.999202 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa667bd9-e2ca-4481-ac3d-1c34078d7dcc service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:21:54.999672 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa667bd9-e2ca-4481-ac3d-1c34078d7dcc service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:21:55.000086 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa667bd9-e2ca-4481-ac3d-1c34078d7dcc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:21:55.002730 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-aa667bd9-e2ca-4481-ac3d-1c34078d7dcc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:21:55.014990 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa667bd9-e2ca-4481-ac3d-1c34078d7dcc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:21:55.016634 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa667bd9-e2ca-4481-ac3d-1c34078d7dcc service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:21:55.027924 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1442/2889] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:21:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:22:04.012297 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=23 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:22:05.022710 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=33 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:22:12.013321 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55505033-2e46-4268-821a-7e0a934efea9 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:22:12.047120 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55505033-2e46-4268-821a-7e0a934efea9 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:22:12.053557 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cef47e-c3c4-4379-b123-1458720d1c47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2409cb7fc4373a74ddfb48d501d30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:22:12.054268 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cef47e-c3c4-4379-b123-1458720d1c47 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:22:12.054554 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cef47e-c3c4-4379-b123-1458720d1c47 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:22:12.054832 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cef47e-c3c4-4379-b123-1458720d1c47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:22:12.056605 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-77cef47e-c3c4-4379-b123-1458720d1c47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:22:12.082834 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cef47e-c3c4-4379-b123-1458720d1c47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:22:12.084572 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cef47e-c3c4-4379-b123-1458720d1c47 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:22:12.096239 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1448/2890] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:22:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:22:22.091371 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=18 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:23:41.544300 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa667bd9-e2ca-4481-ac3d-1c34078d7dcc service placement] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:23:41.546053 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-aa667bd9-e2ca-4481-ac3d-1c34078d7dcc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:23:41.559359 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa667bd9-e2ca-4481-ac3d-1c34078d7dcc service placement] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:23:41.564288 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b26e07-7285-4dfc-9048-efd2c1367505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:23:41.564963 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b26e07-7285-4dfc-9048-efd2c1367505 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:23:41.565244 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b26e07-7285-4dfc-9048-efd2c1367505 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:23:41.565518 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b26e07-7285-4dfc-9048-efd2c1367505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:23:41.567930 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c2b26e07-7285-4dfc-9048-efd2c1367505 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:23:41.579148 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b26e07-7285-4dfc-9048-efd2c1367505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:23:41.580594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b26e07-7285-4dfc-9048-efd2c1367505 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:23:41.589820 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1443/2891] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:23:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:23:42.695070 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cef47e-c3c4-4379-b123-1458720d1c47 service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:23:42.705739 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cef47e-c3c4-4379-b123-1458720d1c47 service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:23:42.709930 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09603c54-ed7e-40fb-a408-7d6b6b8c3cf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:23:42.710585 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09603c54-ed7e-40fb-a408-7d6b6b8c3cf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:23:42.710896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09603c54-ed7e-40fb-a408-7d6b6b8c3cf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:23:42.711165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09603c54-ed7e-40fb-a408-7d6b6b8c3cf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:23:42.713048 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-09603c54-ed7e-40fb-a408-7d6b6b8c3cf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:23:42.727372 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09603c54-ed7e-40fb-a408-7d6b6b8c3cf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:23:42.728879 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09603c54-ed7e-40fb-a408-7d6b6b8c3cf7 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:23:42.740496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1449/2892] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:23:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:23:43.929649 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b26e07-7285-4dfc-9048-efd2c1367505 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:23:43.939125 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b26e07-7285-4dfc-9048-efd2c1367505 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:23:43.943785 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a135b-ef56-4595-8d75-d2023b8cb6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:23:43.944496 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a135b-ef56-4595-8d75-d2023b8cb6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:23:43.944747 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a135b-ef56-4595-8d75-d2023b8cb6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:23:43.945052 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a135b-ef56-4595-8d75-d2023b8cb6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:23:43.947009 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-333a135b-ef56-4595-8d75-d2023b8cb6b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:23:43.961949 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a135b-ef56-4595-8d75-d2023b8cb6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:23:43.963396 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a135b-ef56-4595-8d75-d2023b8cb6b0 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:23:43.975045 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1444/2893] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:23:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:23:52.735552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:23:53.970601 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=12 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:23:55.515329 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09603c54-ed7e-40fb-a408-7d6b6b8c3cf7 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:23:55.524626 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09603c54-ed7e-40fb-a408-7d6b6b8c3cf7 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:23:55.528769 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252335a5-341e-4f5e-82dd-1371c1281b63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1f11a4b8554a98b7626a82f01888c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:23:55.529682 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252335a5-341e-4f5e-82dd-1371c1281b63 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:23:55.529971 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252335a5-341e-4f5e-82dd-1371c1281b63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:23:55.530250 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252335a5-341e-4f5e-82dd-1371c1281b63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:23:55.532089 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-252335a5-341e-4f5e-82dd-1371c1281b63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:23:55.626089 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252335a5-341e-4f5e-82dd-1371c1281b63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:23:55.628612 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252335a5-341e-4f5e-82dd-1371c1281b63 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:23:55.654218 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1450/2894] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:23:58.744301 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a135b-ef56-4595-8d75-d2023b8cb6b0 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:23:58.756582 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a135b-ef56-4595-8d75-d2023b8cb6b0 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:23:58.762313 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ff1e4-63ed-4b9d-8835-5d015ddbc12b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:23:58.762974 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ff1e4-63ed-4b9d-8835-5d015ddbc12b service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:23:58.763259 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ff1e4-63ed-4b9d-8835-5d015ddbc12b service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:23:58.763537 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ff1e4-63ed-4b9d-8835-5d015ddbc12b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:23:58.765460 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ac7ff1e4-63ed-4b9d-8835-5d015ddbc12b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:23:58.778070 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ff1e4-63ed-4b9d-8835-5d015ddbc12b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:23:58.779893 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ff1e4-63ed-4b9d-8835-5d015ddbc12b service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:23:58.790988 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1445/2895] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:23:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:24:01.690781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252335a5-341e-4f5e-82dd-1371c1281b63 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:24:01.692344 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-252335a5-341e-4f5e-82dd-1371c1281b63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:24:01.705451 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252335a5-341e-4f5e-82dd-1371c1281b63 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:24:01.710080 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:24:01.710996 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:24:01.711280 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:24:01.711558 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] PATH_INFO: `/v3/users/2321028b4d4f4734afd1ce674931a4d3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:24:01.712746 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2321028b4d4f4734afd1ce674931a4d3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:24:01.714807 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:24:01.763273 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2321028b4d4f4734afd1ce674931a4d3'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:01.763924 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2321028b4d4f4734afd1ce674931a4d3'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:01.772261 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2321028b4d4f4734afd1ce674931a4d3'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:01.784139 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2321028b4d4f4734afd1ce674931a4d3'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:01.784781 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2321028b4d4f4734afd1ce674931a4d3'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:01.796659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1451/2896] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:24:01 2022] DELETE /identity/v3/users/2321028b4d4f4734afd1ce674931a4d3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:24:01.989647 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ff1e4-63ed-4b9d-8835-5d015ddbc12b service placement] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:24:02.002262 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ff1e4-63ed-4b9d-8835-5d015ddbc12b service placement] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:24:02.064621 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:24:02.065918 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:24:02.066326 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:24:02.066627 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] PATH_INFO: `/v3/projects/aaff1fa03a194074aa14fae9db94cefb` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:24:02.068881 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaff1fa03a194074aa14fae9db94cefb)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:24:02.070927 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:24:02.130271 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaff1fa03a194074aa14fae9db94cefb', 'request_id': 'req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.137151 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaff1fa03a194074aa14fae9db94cefb', 'request_id': 'req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.143594 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaff1fa03a194074aa14fae9db94cefb', 'request_id': 'req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.158678 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] The token cache is being invalidate because project aaff1fa03a194074aa14fae9db94cefb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:24:02.158976 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.159673 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.161307 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1446/2897] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:24:01 2022] DELETE /identity/v3/projects/aaff1fa03a194074aa14fae9db94cefb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:24:02.165434 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:24:02.166896 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:24:02.180675 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5615a-20ed-4900-ac0d-8d6bcc2628d9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:24:02.215383 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:24:02.244165 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:24:02.245063 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:24:02.245346 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:24:02.245659 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] PATH_INFO: `/v3/users/ead485491f7d4c4b85e2974d377996c2` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:24:02.247018 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ead485491f7d4c4b85e2974d377996c2)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:24:02.248983 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:24:02.307496 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ead485491f7d4c4b85e2974d377996c2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.308246 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ead485491f7d4c4b85e2974d377996c2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.317125 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ead485491f7d4c4b85e2974d377996c2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.328882 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ead485491f7d4c4b85e2974d377996c2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.329536 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ead485491f7d4c4b85e2974d377996c2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.342066 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1452/2898] 10.222.0.79 () {58 vars in 1223 bytes} [Mon Jul 4 10:24:02 2022] DELETE /identity/v3/users/ead485491f7d4c4b85e2974d377996c2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:24:02.535357 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:24:02.536863 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:24:02.553129 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa5c56-1e7c-4ed9-b3bf-e60768770f24 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:24:02.610241 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4ef77-01a0-47c8-af9d-68ba6027a604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5875b90a72824e1280ee921aaccb7829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca4e68f3b1bd4167b76136fe537c9f10', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:24:02.611121 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4ef77-01a0-47c8-af9d-68ba6027a604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:24:02.611403 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4ef77-01a0-47c8-af9d-68ba6027a604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:24:02.611681 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4ef77-01a0-47c8-af9d-68ba6027a604 admin admin] PATH_INFO: `/v3/projects/69bd956122ce4cc8b7d4d97f4f0c1868` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:24:02.612855 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4ef77-01a0-47c8-af9d-68ba6027a604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69bd956122ce4cc8b7d4d97f4f0c1868)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:24:02.614804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4ef77-01a0-47c8-af9d-68ba6027a604 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:24:02.636497 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:24:02.664689 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-64c4ef77-01a0-47c8-af9d-68ba6027a604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69bd956122ce4cc8b7d4d97f4f0c1868', 'request_id': 'req-64c4ef77-01a0-47c8-af9d-68ba6027a604'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.670804 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-64c4ef77-01a0-47c8-af9d-68ba6027a604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69bd956122ce4cc8b7d4d97f4f0c1868', 'request_id': 'req-64c4ef77-01a0-47c8-af9d-68ba6027a604'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.676398 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-64c4ef77-01a0-47c8-af9d-68ba6027a604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69bd956122ce4cc8b7d4d97f4f0c1868', 'request_id': 'req-64c4ef77-01a0-47c8-af9d-68ba6027a604'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.691014 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-64c4ef77-01a0-47c8-af9d-68ba6027a604 admin admin] The token cache is being invalidate because project 69bd956122ce4cc8b7d4d97f4f0c1868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 10:24:02.691302 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-64c4ef77-01a0-47c8-af9d-68ba6027a604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.691922 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-64c4ef77-01a0-47c8-af9d-68ba6027a604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 10:24:02.693458 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1447/2899] 10.222.0.79 () {58 vars in 1232 bytes} [Mon Jul 4 10:24:02 2022] DELETE /identity/v3/projects/69bd956122ce4cc8b7d4d97f4f0c1868 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 10:24:12.338569 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:24:12.688319 dv-cinder-848418-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 10:27:41.840550 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 10:27:41.842231 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:27:41.915339 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957c9d6-1a06-4c46-8c0d-7139486e6d48 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 10:27:41.978375 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7ad0f-0be5-47e1-a62b-5cb1fd310f38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dc48318ab7944c197bf5f7673511931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f32306262e8744eab4bdc6d28e24e6d0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 10:27:41.979035 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7ad0f-0be5-47e1-a62b-5cb1fd310f38 service placement] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 10:27:41.979320 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7ad0f-0be5-47e1-a62b-5cb1fd310f38 service placement] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 10:27:41.979595 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7ad0f-0be5-47e1-a62b-5cb1fd310f38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 10:27:41.981552 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-03f7ad0f-0be5-47e1-a62b-5cb1fd310f38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 10:27:41.993570 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7ad0f-0be5-47e1-a62b-5cb1fd310f38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 10:27:41.995201 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7ad0f-0be5-47e1-a62b-5cb1fd310f38 service placement] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 10:27:42.005281 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1453/2900] 10.222.0.79 () {60 vars in 1365 bytes} [Mon Jul 4 10:27:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 10:27:52.001248 dv-cinder-848418-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=58 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}